3D simulation video remote synchronous playing method and playing system

Abstract

The invention discloses a 3D simulation video remote synchronous playing method. The method comprises the following steps: S1, acquiring action information and state information of control equipment in real time; s2, preprocessing the action information and the state information, and generating a local 3D simulation video in real time; s3, extracting action parameters, scene change parameters and result information from the local 3D simulation video; s4, synchronously generating a remote 3D simulation video by using the action parameters, the scene change parameters and the result information; data transmitted to different places are only action parameters, scene change parameters and result information, the transmission bandwidth and transmission delay of the 3D video are effectively reduced, the action parameters, the scene change parameters and the result information can be sent only through narrow-band wireless transmission, the purpose of local and different-place synchronous playing of the high-definition 3D simulation video is achieved, and the high-definition 3D simulation video playing efficiency is improved. The technical processing complexity can be reduced, the stability and reliability of the system are improved, and the image transmission capability required by system operation is achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of high-definition 3D video image processing and wireless transmission, and specifically relates to a 3D simulation video remote synchronous playback method and playback system. Background Art

[0002] In the operating system, 3D videos of the operation scenes need to be played synchronously in real time at two locations. The operating system generates 3D simulation videos in real time based on the input data, and plays them synchronously locally and remotely. For example, in the scenario of rocket launcher firing teaching, the trainer carries control equipment such as simulated rocket launcher props for firing training. By collecting the motion information and status information of the simulated rocket launcher props, the operating system used for rocket launcher firing teaching can generate 3D simulation videos in real time for the trainers to watch, and can also transmit the generated 3D simulation videos to remote locations for instructors to watch and provide teaching guidance.

[0003] When multiple operating systems are operating at the same time, real-time wireless video transmission will cause mutual interference due to the large bandwidth occupied, and the stability and reliability of the system cannot be guaranteed. Summary of the invention

[0004] The object of the present invention is to provide a 3D simulation video remote synchronous playback method and playback system to solve at least one of the above-mentioned technical problems.

[0005] To achieve the above object, the technical solution adopted by the present invention is: A method for synchronously playing 3D simulation videos in different locations, comprising the following steps: S1, real-time collection of action information and status information of the control device; S2, preprocessing the action information and state information, and generating local 3D simulation video in real time; S3, extracting action parameters, scene change parameters and result information from the local 3D simulation video; S4. Synchronously generate a remote 3D simulation video using the action parameters, scene change parameters and result information.

[0006] The data transmitted to a remote location by the present invention are only action parameters, scene change parameters and result information, which effectively reduces the transmission bandwidth and transmission delay of 3D video. The action parameters, scene change parameters and result information can be sent only by narrowband wireless transmission, so as to achieve the purpose of local and remote synchronous playback of high-definition 3D simulation video, reduce the complexity of technical processing, increase the stability and reliability of the system, and achieve the image transmission capability required by the system operation. The present invention first generates a local 3D simulation video, and then extracts deterministic action parameters, scene change parameters and result information from the local 3D simulation video to generate a remote 3D simulation video, thereby ensuring that the local 3D simulation video and the remote 3D simulation video are consistent, avoiding calculation errors caused by preprocessing action information and status information locally and remotely, resulting in inconsistent generated 3D simulation videos.

[0007] Furthermore, the status information of the control device includes before firing, during firing and after firing.

[0008] Furthermore, the state information of the control device is determined based on whether the control device has performed a firing operation. The state information of the control device before performing a firing operation is before firing, the state information when performing a firing operation is during firing, and the state information after performing a firing operation is after firing.

[0009] Furthermore, in step S1, only the action information of the control device before and during firing is collected.

[0010] Furthermore, when the state information is before firing and during firing, a local 3D simulation video corresponding to the collected action information and state information is generated in real time according to the action information and state information collected in step S1.

[0011] Furthermore, when the status information is after firing, the action information and status information of the control device after the firing operation are simulated and calculated according to the action information collected when the control device performs the firing operation, so as to generate a local 3D simulation video in real time; when performing the simulation calculation, affected by various factors, some errors may also occur when different devices perform simulation calculations on the same input data. Therefore, the present invention extracts deterministic action parameters, scene change parameters and result information from the local 3D simulation video for generating a remote 3D simulation video, thereby ensuring that the 3D simulation videos generated locally and remotely are consistent.

[0012] Furthermore, the action parameters include a second coordinate parameter, a second posture parameter and a second viewing angle parameter; the second coordinate parameter, the second posture parameter and the second viewing angle parameter can directly generate a corresponding remote 3D simulation video.

[0013] Furthermore, the result information includes an action result and a collision result. The action result is the complete motion trajectory of the shell after firing, and the collision result is a simulation calculation of whether the shell collides with the target after the control device is fired, thereby determining whether the shell hits the target.

[0014] The present invention also provides a 3D simulation video remote synchronous playback system, based on the above playback method, comprising: Motion collection module, used to collect the motion information of the control device; Status detection module, used to collect and control device status information; A first video simulation module, used for generating local 3D simulation video in real time; A control module, connected to the motion acquisition module, the state detection module and the first video simulation module, for data processing; A first display module, connected to the first video simulation module, for displaying a local 3D simulation video generated in real time by the first video simulation module; A wireless transmission module, connected to the control module and used for wireless data transmission; A second video simulation module, which is connected to the wireless transmission module and is used to generate a remote 3D simulation video in real time; The second display module is connected to the second video simulation module and is used to display the remote 3D simulation video generated in real time by the second video simulation module.

[0015] Compared with the prior art, the present invention has the following beneficial effects: (1) The data transmitted to a remote location by the present invention are only action parameters, scene change parameters and result information, which effectively reduces the transmission bandwidth and transmission delay of 3D video. The action parameters, scene change parameters and result information can be sent only by narrowband wireless transmission, so as to achieve the purpose of local and remote synchronous playback of high-definition 3D simulation video, reduce the complexity of technical processing, increase the stability and reliability of the system, and achieve the image transmission capability required by the system operation; (2) The present invention first generates a local 3D simulation video, and then extracts deterministic action parameters, scene change parameters and result information from the local 3D simulation video to generate a remote 3D simulation video, thereby ensuring that the local 3D simulation video and the remote 3D simulation video are consistent, avoiding calculation errors caused by pre-processing the action information and status information locally and remotely, resulting in inconsistent generated 3D simulation videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a method for synchronously playing 3D simulation videos in different locations in this embodiment; Figure 2This is a wireframe diagram of a 3D simulation video remote synchronous playback system in this embodiment; Figure 3 This is a screenshot of the remote 3D simulation video generated before firing in this embodiment; Figure 4 This is a screenshot of the remote 3D simulation video generated after firing in this embodiment. DETAILED DESCRIPTION

[0017] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, this embodiment provides a method for synchronously playing a 3D simulation video in different locations, comprising the following steps: S1, real-time collection of action information and status information of the control device, with a collection time interval of less than 5ms; In this embodiment, the control device is a combat training simulation prop, and the action information includes a first coordinate parameter, a first posture parameter, and a first viewing angle parameter. The first coordinate parameter is a coordinate parameter of the control device collected in real time, the first posture parameter is a posture parameter of the control device collected in real time, and the first viewing angle parameter is a viewing angle parameter of the control device collected in real time, wherein the posture parameter refers to the direction and tilt angle of the control device; In this embodiment, the motion information of the control device can be collected by a motion collection module, and the motion collection module can include a posture measurement device, and the first coordinate parameter, the first posture parameter and the first viewing angle parameter of the control device are collected by the posture measurement device; the posture measurement device can be a posture sensor, and in this embodiment, one or more posture sensors can be combined to collect the first coordinate parameter, the first posture parameter and the first viewing angle parameter, and the posture sensor can be an accelerometer, a gyroscope, a magnetometer, an optical posture sensor or an ultrasonic sensor; for example, when the posture sensor adopts a gyroscope, the gyroscope can measure the direction and tilt angle of the control device, that is, the gyroscope can measure the first posture parameter of the control device. In this embodiment, the combat training simulation prop can be a simulated rocket launcher prop, and the barrel direction of the simulated rocket launcher prop is used as the viewing angle direction. The viewing angle of the barrel can be tracked by the gyroscope, so as to obtain the first viewing angle parameter; in this embodiment, an initial coordinate can be set first, and the movement of the control device can be detected by the gyroscope to obtain subsequent coordinates on the basis of the initial coordinates, and then the first coordinate parameter is obtained. The initial coordinate can be set manually.

[0019] In this embodiment, the status information of the control device includes before firing, during firing and after firing; the control device is a combat training simulation prop, and the combat training simulation prop can be a simulated rocket launcher prop. The simulated rocket launcher prop is used to simulate a real rocket launcher, and the rocket launcher can be divided into three states according to whether it is fired, namely, before the rocket launcher is fired, during firing, and after firing. In this embodiment, firing refers to pressing the launch switch of the rocket launcher to fire the rocket launcher; in this embodiment, a firing switch is provided on the control device. When the firing switch is not pressed, the control device is in the pre-firing state, when the firing switch is pressed, the control device is in the firing state, and when the firing switch is pressed, the control device is in the post-firing state. The set firing switch can be used to collect the status information of the control device.

[0020] S2, preprocessing the action information and state information, and generating local 3D simulation video in real time; In this embodiment, after the firing switch of the control device, i.e., the simulated rocket launcher prop, is pressed, the rocket launcher can be simulated to fire a shell, and a corresponding 3D simulation video of the shell firing can be generated.

[0021] In this embodiment, preprocessing the action information and the state information to generate a local 3D simulation video in real time includes: When the status information is before firing or during firing, the local 3D simulation video is directly generated in real time through the action information; When the status information is after firing, the action information after firing, the movement trajectory of the shell and the collision result of the shell are simulated and calculated according to the action information at the time of firing. By simulating and calculating the action information after firing, the movement trajectory of the shell and the collision result of the shell, a local 3D simulation video is generated in real time. The collision result of the shell is whether the shell hits the target. By simulating the calculated action information after firing to generate a local 3D simulation video, the jitter of the perspective when the shell is fired can be simulated, and the recoil of the rocket launcher when fired can be reflected, thereby achieving a more realistic training effect.

[0022] In this embodiment, an animation driver is used to generate a local 3D simulation video. The animation driver generates a corresponding 3D simulation video by inputting action information, state information, and simulating and calculating the action information after firing, the movement trajectory of the shell, and the collision result of the shell.

[0023] In this embodiment, the animation driver used to generate local 3D simulation video is a 3D shooting game program. In this embodiment, the 3D shooting game program is developed through a game engine, and the game engine can be one of Unity, UnrealEngine, CryEngine and Godot; in this embodiment, the 3D shooting game program also needs to perform 3D modeling and animation production. During 3D modeling, 3D modeling software is used to create scene environments and props. The 3D modeling software can be one of Blender, Maya or 3ds Max. Props include simulated rocket launchers, artillery shells and targets, etc.

[0024] S3, extracting action parameters, scene change parameters and result information from the local 3D simulation video; In this embodiment, the action parameters include a second coordinate parameter, a second posture parameter and a second viewing angle parameter. The second coordinate parameter is a coordinate parameter of the control device extracted from the local 3D simulation video, the second posture parameter is a posture parameter of the control device extracted from the local 3D simulation video, and the second viewing angle parameter is a viewing angle parameter of the control device extracted from the local 3D simulation video.

[0025] In this embodiment, the result information includes an action result and collision information. The action result is the movement trajectory of the shell, and the collision information is whether the shell hits the target.

[0026] In this embodiment, the scene change parameters include weather parameters and scene environment parameters. Before generating a 3D simulation video, the corresponding weather and scene environment in the 3D simulation video can be set. The weather includes sunny days, cloudy days, rainy days, snowy days, etc., and the scene environment includes mountains, hills, towns, deserts, grasslands, farmlands, etc.; the weather parameter is the type of weather. For example, before generating a local 3D simulation video, the weather is set to sunny, and the weather parameters extracted from the local 3D simulation video are sunny; the scene environment parameter is the type of scene environment. For example, before generating a local 3D simulation video, the scene environment is set to mountains, and the scene environment parameters extracted from the local 3D simulation video are mountains.

[0027] S4. Synchronously generate a remote 3D simulation video using the action parameters, scene change parameters and result information.

[0028] In this embodiment, an animation driver is used to generate a remote 3D simulation video. The animation driver generates a corresponding 3D simulation video through input action parameters, scene change parameters and result information.

[0029] The data transmitted to a remote location by the present invention are only action parameters, scene change parameters and result information. Compared with directly transmitting video, the amount of data transmitted is reduced from 10Mbps to no more than 20Kbps, which effectively reduces the transmission bandwidth and transmission delay of 3D video. Only narrowband wireless transmission is used to send action parameters, scene change parameters and result information, so as to achieve the purpose of local and remote synchronous playback of high-definition 3D simulation video, reduce the complexity of technical processing, increase the stability and reliability of the system, and achieve the image transmission capability required by system operation. The present invention first generates a local 3D simulation video, and then extracts deterministic action parameters, scene change parameters and result information from the local 3D simulation video to generate a remote 3D simulation video, thereby ensuring that the local 3D simulation video and the remote 3D simulation video are consistent, avoiding calculation errors caused by preprocessing action information and state information locally and remotely, resulting in inconsistent generated 3D simulation videos.

[0030] Furthermore, the status information of the control device includes before firing, during firing and after firing. The control device is a combat training simulation prop. The combat training simulation prop can be a simulated rocket launcher prop. The simulated rocket launcher prop is used to simulate a real rocket launcher. The rocket launcher can be divided into three states according to whether it is fired, namely, before the rocket launcher is fired, during the rocket launcher firing, and after the rocket launcher is fired.

[0031] Furthermore, the state information of the control device is determined based on whether the control device has performed a firing operation. The state information of the control device before performing the firing operation is before firing, the state information when performing the firing operation is during firing, and the state information after performing the firing operation is after firing. In the present embodiment, firing refers to pressing the firing switch of the rocket launcher to fire the rocket launcher. In the present embodiment, a firing switch is provided on the control device. When the firing switch is not pressed, the control device is in a pre-firing state. When the firing switch is pressed, the control device is in a during firing state. When the firing switch is pressed, the control device is in a post-firing state. The set firing switch can be used to collect the state information of the control device. For example, when the state information of the control device is before firing, the action parameters, scene change parameters and result information can be used to generate the following information: Figure 3 The monitoring screen shown (screenshot of the remote 3D simulation video) is Figure 3 In the display, it is shown that the shot has not been fired (not fired). When the state information of the control device is after firing, the action parameters, scene change parameters and result information can be used to generate the following Figure 4 The monitoring screen shown (screenshot of the remote 3D simulation video) is Figure 4 The shooting result shows a hit.

[0032] Furthermore, in step S1, only the action information of the control device before and during firing is collected.

[0033] Furthermore, when the state information is before firing and during firing, a local 3D simulation video corresponding to the collected action information and state information is generated in real time according to the action information and state information collected in step S1.

[0034] Furthermore, when the status information is after firing, the motion information and status information of the control device after the firing operation are simulated and calculated based on the motion information collected when the control device performs the firing operation, which is used to generate a local 3D simulation video in real time; when the status information is after firing, the motion trajectory of the artillery shell after the firing operation and the collision result of the artillery shell are simulated and calculated based on the motion information collected when the control device performs the firing operation, which is used to generate a local 3D simulation video.

[0035] When performing simulation calculations, due to the influence of various factors, some errors may occur when different devices perform simulation calculations on the same input data. Therefore, the present invention extracts deterministic action parameters, scene change parameters and result information from the local 3D simulation video to generate a remote 3D simulation video, thereby ensuring that the 3D simulation videos generated locally and remotely are consistent.

[0036] Furthermore, the action parameters include a second coordinate parameter, a second posture parameter and a second viewing angle parameter; the second coordinate parameter, the second posture parameter and the second viewing angle parameter combined with the scene change parameter and the result information can directly generate a corresponding remote 3D simulation video.

[0037] Furthermore, the result information includes an action result and a collision result. The action result is the complete motion trajectory of the shell after firing, and the collision result is a simulation calculation of whether the shell collides with the target after the control device is fired, thereby determining whether the rocket launcher hits the target.

[0038] like Figure 2 As shown, the present invention also provides a 3D simulation video remote synchronous playback system, based on the above playback method, comprising: A motion acquisition module, used to acquire motion information of the control device; preferably, the motion acquisition module may include a posture measurement device, and the motion information of the control device is acquired through the posture measurement device; A state detection module, used to collect state information of the control device; preferably, the state detection module may include a state detection sensor, used to detect whether the control device is fired, so as to determine the state of the control device, and the state detection sensor may be a firing switch; A first video simulation module, for generating a local 3D simulation video in real time; preferably, the first video simulation module may include a video simulation computer, through which a 3D video simulating a real scene is generated in real time, and the video simulation computer stores an animation driver for generating the local 3D simulation video; A control module, connected to the motion acquisition module, the state detection module and the first video simulation module, for data processing; A first display module, connected to the first video simulation module, for displaying a local 3D simulation video generated in real time by the first video simulation module; A wireless transmission module, connected to the control module and used for wireless data transmission; In this embodiment, the wireless transmission module may be a WiFi module or a 4G / 5G module.

[0039] A second video simulation module, which is connected to the wireless transmission module and is used to generate a remote 3D simulation video in real time. The second video simulation module may also include a video simulation computer, which generates a 3D video simulating a real scene in real time through the video simulation computer, and the video simulation computer stores an animation driver for generating the remote 3D simulation video; The second display module is connected to the second video simulation module and is used to display the remote 3D simulation video generated in real time by the second video simulation module.

[0040] The first video simulation module and the second video simulation module are pre-stored with consistent common parameters so that the first video simulation module and the second video simulation module can generate the same 3D simulation video. The common parameters include data required for generating various weather and scene environments. For example, when the weather in the 3D simulation video generated by the first video simulation module is sunny and the scene environment is mountainous, the second video simulation module can also generate the same 3D simulation video with sunny weather and mountainous scene environment.

[0041] It should be noted that although the present invention is disclosed as above by specific embodiments, the above embodiments are not intended to limit the present invention. Ordinary technicians in this field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined in the claims.

Claims

1. A method for synchronously playing 3D simulation videos in different locations, characterized in that: The following steps are involved: S1, real-time collection of action information and status information of the control device; S2, preprocessing the action information and state information, and generating local 3D simulation video in real time; S3, extracting action parameters, scene change parameters and result information from the local 3D simulation video; S4. Synchronously generate a remote 3D simulation video using the action parameters, scene change parameters and result information.

2. The 3D simulation video remote synchronous playback method according to claim 1, characterized in that: The status information of the control device includes before firing, during firing and after firing.

3. The 3D simulation video remote synchronous playback method according to claim 2, characterized in that: The state information of the control device is determined based on whether the control device has performed a firing operation. The state information of the control device before performing a firing operation is before firing, the state information when performing a firing operation is during firing, and the state information after performing a firing operation is after firing.

4. The 3D simulation video remote synchronous playback method according to claim 3, characterized in that: In step S1, only the action information of the control device before and during firing is collected.

5. The method for synchronously playing 3D simulation videos in different locations according to claim 4, characterized in that: When the state information is before firing and during firing, a local 3D simulation video corresponding to the collected action information and state information is generated in real time according to the action information and state information collected in step S1.

6. The method for synchronously playing 3D simulation videos in different locations according to claim 4, characterized in that: When the status information is after firing, the action information and status information of the control device after the firing operation are simulated and calculated according to the action information collected when the control device performs the firing operation, so as to generate the local 3D simulation video in real time.

7. The method for synchronously playing 3D simulation videos in different locations according to claim 6, characterized in that: The action parameters include a second coordinate parameter, a second posture parameter and a second viewing angle parameter.

8. The method for synchronously playing 3D simulation videos in different locations according to claim 6, characterized in that: The result information includes action results and collision results.

9. A 3D simulation video remote synchronous playback system, based on any playback method in claims 1-8, characterized in that: include: Motion collection module, used to collect the motion information of the control device; Status detection module, used to collect and control device status information; A first video simulation module, used for generating local 3D simulation video in real time; A control module, connected to the motion acquisition module, the state detection module and the first video simulation module, for data processing; A first display module, connected to the first video simulation module, for displaying a local 3D simulation video generated in real time by the first video simulation module; A wireless transmission module, connected to the control module and used for wireless data transmission; A second video simulation module, which is connected to the wireless transmission module and is used to generate a remote 3D simulation video in real time; The second display module is connected to the second video simulation module and is used to display the remote 3D simulation video generated in real time by the second video simulation module.

Images