Cross-platform three-dimensional virtual simulation courseware compiling and playing method and system

Abstract

The invention provides a cross-platform three-dimensional virtual simulation courseware compiling and playing method and system. The method comprises the steps of defining a file format, a target file position and an identification mode; creating a user interaction interface, receiving a three-dimensional virtual simulation file generated by a user through operating the user interaction interface configuration and an uploaded resource file, sorting the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, the target file position and the identification mode, and sending the sorted three-dimensional virtual simulation file and the uploaded resource file to the user interaction interface. Packaging the arranged three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file; and reading and analyzing the courseware file, and playing the courseware file according to the configuration of the playing platform and the playing mode selected by the user. According to the embodiment of the invention, as long as the same training content courseware is made once, the purpose of using the same training content courseware on playing platforms with different configurations can be achieved.

Description

Technical Field

[0001] The present application relates to the field of education and training, and in particular to a method and system for compiling and playing cross-platform three-dimensional virtual simulation courseware. Background Art

[0002] In recent years, 3D technology has developed rapidly. In terms of hardware, such as NVIDIA's GPU technology, display screen resolution, and VR / AR applications, in terms of software, 3D immersion engines are already very mature, such as Unreal and Unity, which can be used to develop 3D applications. Its use can give users an immersive experience, because it is applied in more and more fields, such as games, film and television production, industrial facilities, medical treatment, education, etc.

[0003] In the field of education, the application of virtual simulation technology for skills training has the advantages of low cost, avoiding dangerous operations and experiencing reality, which has incomparable advantages over traditional teaching methods. The traditional virtual simulation training course development model is usually developed using 3D rendering engines such as Unity or Unreal. The typical development process includes:

[0004] (1) Business domain experts organize course content requirements and generate documents for the 3D technology development team.

[0005] (2) The 3D technology development team conducts modeling and code development according to the course content requirements. The development work at this stage includes interface production, 3D scene construction, interactive action development, exam content development, network functions, and database interaction development. At this stage, you need to master 3D rendering engine technology, such as C# / C++ development, 3D engine lighting, mapping, interface, action and special effects, and compile it into a strong executable file after completion.

[0006] (3) The 3D technology development team develops corresponding versions of applications for different hardware platforms based on business needs, such as different models of VR, AR and other devices.

[0007] Direct development using 3D technology requires a lot of communication between the business team and the 3D technology development team, and executable courseware can only be implemented by the 3D technology development team. The courseware content is included in the executable program. The technical team needs to write different codes for various hardware platform functions and compile them into executable programs. There is no guarantee that a courseware can run on multiple different platforms. The process of developing courseware requires a lot of communication between business domain experts and the technical development team, which is inefficient. The maintenance of courseware content always requires the participation of the technical team, which is costly. Summary of the invention

[0008] In order to solve one of the above technical defects, a cross-platform three-dimensional virtual simulation courseware compilation and playback method and system are provided in an embodiment of the present application.

[0009] A first aspect of an embodiment of the present application provides a cross-platform three-dimensional virtual simulation courseware compilation and playback method, the method comprising:

[0010] Define file formats, target file locations, and identification methods;

[0011] Creating a user interaction interface, receiving a three-dimensional virtual simulation file and an uploaded resource file generated by a user by operating the user interaction interface configuration, organizing the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method, and packaging the organized three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file;

[0012] The courseware file is read and parsed, and the courseware file is played according to the configuration of the playback platform and the playback mode selected by the user.

[0013] A second aspect of the embodiment of the present application provides a cross-platform three-dimensional virtual simulation courseware compilation system, the system comprising a production module and a playback module;

[0014] The production module is used to define the file format, target file location and identification method, create a user interaction interface, receive the three-dimensional virtual simulation file and the uploaded resource file generated by the user by operating the user interaction interface configuration, organize the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method, and package the organized three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file;

[0015] The playing module is used to read and parse the courseware file, and play the courseware file according to the configuration of the playing platform and the playing mode selected by the user.

[0016] The cross-platform three-dimensional virtual simulation courseware compilation and playback method provided in the embodiment of the present application is adopted, by defining the file format, the target file location and the identification method, and organizing the three-dimensional virtual simulation file and the uploaded resource file, and finally playing the courseware file according to the configuration of the playback platform and the playback method selected by the user, so that the user can directly produce the courseware content, reduce unnecessary developer participation links, and improve work efficiency. In addition, the embodiment of the present application only needs to produce the same training content courseware once, and it can be used on playback platforms with different configurations, reducing the development and work costs of the training courseware when it is used across platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 This is a flow chart of the cross-platform three-dimensional virtual simulation courseware compilation and playback method provided in Example 1 of the present application. DETAILED DESCRIPTION

[0019] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0020] Example 1

[0021] like Figure 1 As shown, this embodiment proposes a cross-platform 3D virtual simulation courseware compilation and playback method, which can be applied to the 3D virtual simulation teaching process. The method can be used to create and play 3D virtual simulation courseware. The method includes:

[0022] S101, define file format, target file location and identification method;

[0023] S102, creating a user interaction interface, receiving a three-dimensional virtual simulation file and an uploaded resource file generated by a user by operating the user interaction interface configuration, organizing the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method, and packaging the organized three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file;

[0024] S103, reading and parsing the courseware file, and playing the courseware file according to the configuration of the playback platform and the playback mode selected by the user.

[0025] Specifically, in this embodiment, the cross-platform three-dimensional virtual simulation courseware compilation and playback method uses the method of separating the training courseware content carrier from the executable program to achieve the purpose of the same courseware being able to run on a variety of different playback platforms, and can be implemented by two parts, the production software and the playback software. First, a standard file format is defined, and then the simulation courseware in this format is exported by the production software. When making the courseware, there is no need to consider the characteristics of the playback platform, and only the visible content to be expressed is described, such as how many steps a certain process has, which tools are needed for each step, how to operate, etc. Then a corresponding production software platform is developed, and the format file is generated on the production software platform. Finally, the playback software on the specified playback platform is developed. On the one hand, the playback software can read the content of the simulation courseware according to the defined format, and on the other hand, it can perform corresponding display and input and output according to the characteristics of the playback platform.

[0026] First, this embodiment defines the file format, target file location and identification method to define the file format dedicated to the cross-platform three-dimensional virtual simulation courseware of this embodiment. In this process, the following files are mainly defined:

[0027] Define the format, target file location and identification method of the 3D model;

[0028] Define the format, target file location, and identification method of the audio content;

[0029] Define the format, target file location, and identification method of the video content;

[0030] Define the format, target file location and identification method of the image content;

[0031] Defines the format of the content information, the location of the target file, and how to identify it.

[0032] The three-dimensional model involved in this embodiment can be used to express various products or parts objects that need training, as well as virtual scenes, such as workshops, factories, etc. In this embodiment, the three-dimensional model includes mesh, texture, and possible animation. All three-dimensional models are required to be in glb format, and all three-dimensional models are recorded as the models part in the target courseware file format. Each three-dimensional model is named and identified by uuid, and duplicate named three-dimensional models are not allowed.

[0033] The audio content involved in this embodiment is in wav format. All audio content is recorded as the audios part in the target courseware file. Each audio content is named and identified with uuid, and audio content with duplicate names is not allowed.

[0034] The video content involved in this embodiment is in MP4 format. All video content is recorded as the videos part in the target courseware file. Each video content is named and identified by a uuid, and duplicate video content is not allowed.

[0035] The picture content involved in this embodiment is in png format, and all picture content is recorded as the pictures part in the target courseware file. Each picture content is named and identified with uuid, and duplicated picture content is not allowed.

[0036] The description content information involved in this embodiment is stored in json format. The description content information will record the following parts:

[0037] Resource reference information, used to record file identification and size, including identification names (uuid) and content sizes (bytes) of models, audios, videos, and pictures.

[0038] Initial scene information, used to record the 3D model used to form the initial scene, the spatial coordinates of the 3D model, including the coordinate values ​​on the x, y, and z axes, and the rotation values ​​(rx, ry, rz) of the 3D model on each coordinate axis;

[0039] Initial camera information, used to record the default viewing angle when loading the scene, including the coordinate values ​​of the camera on the x, y, and z axes, and the rotation direction of the camera on the coordinate axes (rx, ry, rz);

[0040] Tool information is used to record the tools, tools and materials needed in the course. The above tools, tools and materials can be collectively referred to as tools. For each tool object, the corresponding 3D model uuid identifier, the 3D space coordinates (x, y, z) and orientation (rx, ry, rz) of the initialization position are recorded;

[0041] Drawing information, used to record which process drawings are used for reference in the course, and record the uuid identifier corresponding to each process drawing and the name of each process drawing;

[0042] The process description information is used to record the number of processes, the identification name of each process, the number of steps, the identification name of each step, and each process and step action. The process description information can record the total number of processes. Each process has a corresponding uuid identification and name. Each process will also be divided into multiple steps. Each step also has a corresponding uuid identification and name. The action corresponding to the step will be described in the step.

[0043] In the process description information, there are many types of work step actions. Each type will reference different resources and record the information of the action in the process description information. Specifically, it may include the following situations:

[0044] Displacement information is used to record the displacement and rotation of the 3D model. The displacement information will record the UUID identifier of the 3D model corresponding to the displacement action, the 3D spatial coordinates of the start and end positions of the displacement and the direction, and the text description of the displacement action. If other audio content is associated with the displacement action, the displacement information may also record the UUID identifier of the voice content corresponding to the displacement action and the UUID identifier of the sound effect content corresponding to the displacement action.

[0045] Animation information, used to record the animation playback action associated with the 3D model, including the UUID identifier of the 3D model corresponding to the animation, the name of the animation to be played, the text description of the animation playback action, the UUID identifier of the voice content corresponding to the animation playback action, and the UUID identifier of the sound effect content corresponding to the animation playback action;

[0046] Display image information is used to record the user's action of displaying pictures. The action of displaying pictures will record the corresponding image content UUID identifier in the pictures part, the display duration, and the text description of the display image action through the display image information. If the display image action is associated with other audio content, the display image information will also record the corresponding voice content UUID identifier of the display image action and the corresponding sound effect content UUID identifier of the display image action;

[0047] Display video information, used to record the user's action of playing the video, and specifically record the corresponding video content uuid identifier in the videos part;

[0048] Modify the mapping information, which is used to record the action of changing the appearance of the 3D model. For example, modify the surface appearance of a model in the scene. The modified mapping information will record the UUID identifier of the 3D model in the models part, the mesh name of the network object in the 3D model whose appearance is to be modified, the surface image to be modified, and the text description of the modified mapping action. If the modification of the mapping action is also associated with other audio content, the modified mapping information will also record the UUID identifier of the voice content corresponding to the modification of the mapping action and the UUID identifier of the sound effect content corresponding to the modification of the mapping action;

[0049] Mobile camera information is used to record the movement of the camera in the scene and can be used to change the viewing angle. The mobile camera information will specifically record the target position (x, y, z) and orientation (rx, ry, rz) of the camera in the three-dimensional space, and a text description of the mobile camera movement. If other audio content is associated with the mobile camera movement, the mobile camera information will also record the corresponding voice content uuid identifier of the mobile camera movement and the corresponding sound effect content uuid identifier of the mobile camera movement.

[0050] Through the above process, the file format, target file location and identification method unique to this embodiment can be defined, providing basic logic for users to compile 3D virtual simulation courseware. Users can compile 3D virtual simulation files through the production software, and the production software can provide a corresponding interactive interface to provide users with paths for uploading resources, defining actions and other operations, and ultimately output all 3D virtual simulation files and uploaded resource files into the courseware in the above-defined file format.

[0051] In the production software, a user interaction interface can be provided to allow the user to upload three-dimensional model files, audio files, video files or image files, and the uploaded files can be used to create resource reference information.

[0052] The production software also allows the uploaded 3D models to be classified into a primary level through the user interface to distinguish between ordinary 3D models and 3D models belonging to the tool category. If it is a 3D model belonging to the tool category, a secondary classification operation is required to distinguish whether the 3D model is a tool, tooling or material type. This process is used to establish tool information.

[0053] The production software also allows the user interaction interface to define the names of all process drawings used in the course, as well as the picture content uuid identifier in the pictures section corresponding to each process drawing. This process is used to establish drawing information.

[0054] The production software also allows the uploaded 3D model to be dragged, rotated and other moving operations through the user interface to establish the initial scene. After the user completes the scene definition, the position and orientation information of all 3D models in the scene are used to establish the initial scene information.

[0055] The production software also allows the position of the camera to be moved and rotated through a user interface, and previews the visual effects after the camera is moved and rotated. After confirmation, this process can be used to establish initial camera information.

[0056] The production software also allows adding, modifying and deleting processes through the user interface, specifying the name and order of each process. Under each process, one or more steps of the process can be created. Users can specify a name and description for each step.

[0057] Each work step has a corresponding action. The production software also allows the user to define the type of action, the 3D model uuid identifier referenced by the action, the audio content uuid identifier, the video content uuid identifier and the image content uuid identifier, the start and end position information of the action through the user interaction interface, and can provide a display interface for users to preview the action effect. This process can be used to establish process description information.

[0058] After the user completes the above operations through the user interaction interface, the courseware file can be exported. Specifically, first traverse the process description information of the process and the work step action. Generate a voice content corresponding to the process description information of the process and the work step action, and identify the voice content with a uuid. This process can be implemented using speech synthesis technology. Associate the voice content with the resource reference information, that is, write the voice content into the audios part. When the process and the work step action change, update the uuid identifier of the voice content.

[0059] In this embodiment, after the user generates a 3D virtual simulation file and uploads a resource file through the user interaction interface, the 3D virtual simulation file and the uploaded resource file may be sorted, specifically:

[0060] Marking the three-dimensional model, audio content, video content and picture content according to a defined marking method;

[0061] Recording the marked three-dimensional model, audio content, video content and picture content and corresponding occupied resources in the description content information;

[0062] Serialize the description content information into JSON format and obtain the length of the JSON format;

[0063] Write the length of the json format into the target file corresponding to the courseware file in int64 format, and then write the json content itself into the target file corresponding to the courseware file;

[0064] According to the content of the description content information, the three-dimensional model, audio content, video content and picture content are written into the target file corresponding to the courseware file in sequence.

[0065] Through the above process, all the information in the 3D virtual simulation file and the uploaded resource file can be written into the target file, which is convenient for outputting the courseware file.

[0066] The above process is a three-dimensional virtual simulation courseware compilation process. After the courseware is compiled, it needs to be played on the playback platform. In this embodiment, the playback software can view the courseware file and read the information in the courseware file, and then display the courseware file according to the corresponding configuration of the playback platform and interact with the user.

[0067] Specifically, the courseware file is read first, and the playback software first reads the length of the json format in the target file corresponding to the courseware file, then reads the json content itself, and finally obtains the three-dimensional model, audio content, video content and picture content according to the json content itself.

[0068] Establish user interaction resources, including establishing an image display device, a sound playback device, and a user operation device. The user operation device can obtain user input, which may specifically include a user click input device and a user current perspective input device. These devices may vary slightly depending on the configuration of the playback platform and the influence of environmental factors such as the development language. On an ordinary PC, it can correspond to a computer screen, speaker, and mouse, and an ordinary PC cannot provide a user current perspective input device. On most VR devices, the above devices can correspond to helmet displays, speakers, input handles, and the positioning devices that come with the VR device.

[0069] After reading all the three-dimensional models, all the models involved in the initial scene are placed in the correct position according to the position (x, y, z) and orientation (rx, ry, rz) of the relevant models in the three-dimensional space recorded in the initial scene information in the description content information in json format, and the picture is output to the display device. In addition, the position and orientation of the camera recorded in the initial camera information can also be read in the description content information. The initial camera information represents the default user's initial observation angle of the scene, and the correct user angle of view can be established based on the initial camera information. When the initial scene is established, the default lighting conditions can be used so that the user can clearly observe the three-dimensional model in the scene, and a method for adjusting the light intensity in the scene can also be provided through the user interaction interface.

[0070] According to the tool information describing the content information, a tool interface can be established, and a corresponding interaction entrance can be established on the user interaction interface. When the user clicks on the interaction on the tool interface, the 3D models of all tools, tooling, and materials can be displayed for the user to learn.

[0071] A drawing interface can be established based on the drawing information describing the content information, and a corresponding interaction entrance can be established on the user interaction interface. When the user interacts on the drawing interface, all reference drawings can be displayed.

[0072] A user learning interface and an action playback function can be established based on the process description information describing the content information. All process and work step information is read from the process description information describing the content information, and a user learning interface is established based on the process and work step information. In the user learning interface, all process names can be displayed in a list. The list items corresponding to each process name can be expanded or collapsed. When expanded, all work steps corresponding to the process can be displayed in a list. By reading the process description information describing the content information, the action corresponding to each work step can be obtained, including the text description, voice information, three-dimensional model, start and end positions of the action, and an association can be established with the interactive entrance of the work step.

[0073] An action playback function can be established based on the process description information of the description content information. The action playback function corresponds to the above-mentioned description content information, and can be specifically divided into a displacement action playback function, an animation action playback function, an image display action playback function, a video display action playback function, a texture modification action playback function, and a camera movement action playback function.

[0074] The displacement action playback function reads the 3D model corresponding to the displacement action, the 3D spatial coordinates and orientation of the action start and end positions, the action description information, the action voice file, and the optional action sound effects from the process description information. When the action starts, the 3D model is placed at the start position and moves to the end position by interpolation. The text information corresponding to the displacement action can be displayed on the image display device in the form of subtitles, and the corresponding voice information and sound effect information can be output to the sound playback device.

[0075] The animation action playback function reads the 3D model corresponding to the animation action, the name of the animation, the description information of the animation action, the voice file of the animation action, and the optional action sound effects from the process description information. The animation in the 3D model can be retrieved by the name of the animation and played on the image display device. The text information corresponding to the animation action can be displayed on the image display device in the form of subtitles, and the corresponding voice information and sound effect information can be output to the sound playback device.

[0076] The display image action playback function reads the image content UUID identifier corresponding to the display image action, the description information of the display image action, the voice content UUID identifier of the display image action, and the optional action sound effect content UUID identifier from the process description information. The image content is displayed on the image display device for a period of time until the user clicks on the interaction and closes the screen. The text information corresponding to the display image action can be displayed on the image display device in the form of subtitles, and the corresponding voice information and sound effect information can be output to the sound playback device.

[0077] The display video action playback function reads the video content uuid identifier corresponding to the display video action from the process description information, and displays the video content on the image display device until the video is finished playing or the user clicks to interact and closes the screen.

[0078] The function of modifying the mapping action playback reads the 3D model UUID identifier corresponding to the modification mapping action, the mesh name of the 3D model material to be modified, the target image UUID identifier to be modified into, the description information of the modification mapping action, the voice content UUID identifier of the modification mapping action, and the optional action sound effect content UUID identifier from the process description information. The material of the mesh corresponding to the 3D model is modified into the target image. The text information corresponding to the modification mapping action can be displayed on the image display device in the form of subtitles, and the corresponding voice information and sound effect information can be output to the sound playback device.

[0079] The mobile camera action playback function reads the three-dimensional spatial coordinates of the camera target position corresponding to the mobile camera action from the process description information, and then interpolates the image camera from the current position to the target position. The movement time can be set according to the actual situation. The text information corresponding to the mobile camera action can be displayed on the image display device in the form of subtitles, and the corresponding voice information and sound effect information can be output to the sound playback device.

[0080] The user can conduct interactive learning on the learning interface. When the user clicks a work step on the learning interface, the actions involved in the work step can be called for playback.

[0081] In some optional embodiments, an assessment link may also be set. Specifically, the description content information also includes test information for recording test questions, and the three-dimensional virtual simulation file also includes a test file. An assessment interface is displayed in an image display device. Option information selected by a user through the assessment interface is obtained. The option information is compared with the pre-stored answer information. An assessment result is obtained according to the comparison result, and the assessment result is displayed in the assessment interface.

[0082] This embodiment defines the file format, the target file location and the identification method, and organizes the three-dimensional virtual simulation file and the uploaded resource file, and finally plays the courseware file according to the configuration of the playback platform and the playback method selected by the user, so that the user can directly create the courseware content, reduce unnecessary developer participation links, and improve work efficiency. In addition, the embodiment of the application only needs to create the same training content courseware once, and it can be used on playback platforms with different configurations, reducing the development and work costs of the training courseware when it is used across platforms.

[0083] Taking a college's EMU virtual simulation training project as an example, the production software can be developed by the technical team using Unity 3D technology and delivered to the business team for use. The production software can run in a personal computer environment with a Windows operating system, and has a WYSIWYG visual operation interface. Business personnel can upload 3D models, define all work steps such as the process flow of EMU assembly and debugging, define test questions, and package and output them as custom lprj format courseware files, which can be transferred to the playback software offline.

[0084] In this project, a courseware running module was implemented that is compatible with three hardware platform environments, including: personal computer, zSpace VR all-in-one machine, and LinkVR large screen.

[0085] The courseware running program in the personal computer environment will output the virtual simulation scene picture on the screen, display text content and play voice introduction, and accept user input through the mouse and keyboard.

[0086] The courseware program of the zSpace VR all-in-one machine outputs virtual simulation scene images on the screen, displays text content and plays voice introductions. When the user wears the glasses that come with the device, they can see the three-dimensional scene images. The courseware program accepts user input on this device through the mouse, keyboard and stylus.

[0087] The courseware running program in the LinkVR large-screen environment outputs virtual simulation scene images on the large screen, displays text content, and plays voice introductions. When the user wears the positioning glasses that come with the device, the large-screen image will automatically switch the output perspective according to the current position of the glasses. The courseware running program accepts user input through the handle on this device.

[0088] In this project, business personnel used the courseware production platform to produce a total of 33 coursewares, covering different functions such as assembly and debugging of EMUs. Business personnel did not need to understand several different hardware platforms when producing courseware. All courseware content was produced only once and then published to the three different hardware platforms of this project and ran normally.

[0089] Example 2

[0090] Corresponding to Example 1, this embodiment proposes a cross-platform three-dimensional virtual simulation courseware compilation system, which includes a production module and a playback module;

[0091] The production module is used to define the file format, target file location and identification method, create a user interaction interface, receive the three-dimensional virtual simulation file and the uploaded resource file generated by the user by operating the user interaction interface configuration, organize the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method, and package the organized three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file;

[0092] The playing module is used to read and parse the courseware file, and play the courseware file according to the configuration of the playing platform and the playing mode selected by the user.

[0093] The working process and principle of the cross-platform three-dimensional virtual simulation courseware compilation system proposed in this embodiment can refer to the contents recorded in Example 1, and will not be repeated in this embodiment.

[0094] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0095] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0096] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A cross-platform three-dimensional virtual simulation courseware compilation and playback method, characterized in that: The method comprises: Define file formats, target file locations, and identification methods; Creating a user interaction interface, receiving a three-dimensional virtual simulation file and an uploaded resource file generated by a user by operating the user interaction interface configuration, organizing the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method, and packaging the organized three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file; The courseware file is read and parsed, and the courseware file is played according to the configuration of the playback platform and the playback mode selected by the user.

2. The method according to claim 1, characterized in that The process of defining the file format, the target file location and the identification method includes: Define the format, target file location and identification method of the 3D model; Define the format, target file location, and identification method of the audio content; Define the format, target file location, and identification method of the video content; Define the format, target file location and identification method of the image content; Defines the format of the content information, the location of the target file, and how to identify it.

3. The method according to claim 2, characterized in that The description content information includes: Resource reference information used to record file identification and size; Initial scene information used to record the three-dimensional model used to form the initial scene, the spatial coordinates of the three-dimensional model, and the rotation value of the three-dimensional model on each coordinate axis; Used to record the initial camera information of the default viewing angle when loading the scene, where the default viewing angle is generated by the spatial coordinates of the camera and the rotation orientation values ​​of the camera on each coordinate axis; Tool information for recording the identification of the three-dimensional model belonging to the tool class, the spatial coordinates of the initialization position, and the orientation; Used to record the drawing information of process drawing identification; Used to record the number of processes, the identification name of each process, the number of steps, the identification name of each step, and the process description information of each process and step action.

4. The method according to claim 3, characterized in that The process description information includes: Used to record the movement of the 3D model, the start and end positions of the 3D model displacement, and the spatial coordinates and orientation of the 3D model displacement information; Used to record animation information associated with animation actions of a 3D model; Display image information used to record image actions and image display duration; Display video information for recording video actions; Used to record the modified map information of the action that changes the appearance of the 3D model; Used to record the mobile camera information of the camera space coordinates and orientation movements.

5. The method according to claim 4, characterized in that The uploaded resource files include one or more of three-dimensional model files, audio files, video files or image files.

6. The method according to claim 5, characterized in that When the uploaded resource file is a three-dimensional model file, a first-level classification operation performed by the user on the three-dimensional model file through the user interaction interface is received; when there is a three-dimensional model file of the tool type that is first-level classified among the three-dimensional model files, a second-level classification operation is performed on the three-dimensional model of the tool type and associated with the tool information.

7. The method according to claim 6, characterized in that The three-dimensional virtual simulation file includes one or more of a process drawing file, an initialization scene file, an initial camera file or a process action file.

8. The method according to claim 7, characterized in that When the three-dimensional virtual simulation file is a process drawing file, receiving a process drawing name and an identifier defined by a user, and associating them with the drawing information; When the three-dimensional virtual simulation file is an initialization scene file, receiving a user's movement operation on the three-dimensional model, establishing an initialization scene through the movement operation, and associating it with the initial scene information; When the three-dimensional virtual simulation file is an initial camera file, receiving a user's camera movement operation, generating a camera viewing angle and a preview effect through the movement operation, and associating them with the initial camera information; When the three-dimensional virtual simulation file is a process action file, it receives the user's operation of adding, modifying, deleting or creating a step of the process and associates it with the process description information.

9. The method according to claim 8, characterized in that The process of arranging the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method includes: Marking the three-dimensional model, audio content, video content and picture content according to a defined marking method; Recording the marked three-dimensional model, audio content, video content and picture content and corresponding occupied resources in the description content information; Serialize the description content information into JSON format and obtain the length of the JSON format; Write the length of the json format into the target file corresponding to the courseware file in int64 format, and then write the json content itself into the target file corresponding to the courseware file; According to the content of the description content information, the three-dimensional model, audio content, video content and picture content are written into the target file corresponding to the courseware file in sequence.

10. The method according to claim 9, characterized in that The process of packaging the classified three-dimensional virtual simulation files and the uploaded resource files to generate courseware files includes: Traverse the process description information of the process and work step actions; Generate a voice content corresponding to the process description information of the process and the work step action, and mark the voice content; Associating the voice content with the resource reference information; When the process and step actions are changed, the identification of the voice content is updated.

11. The method according to claim 10, characterized in that The process of reading and parsing the courseware file and playing the courseware file according to the configuration of the playback platform and the playback mode selected by the user includes: Read and parse the courseware file; Establishing interactive resources of image display device, sound playback device and user operation device according to the configuration of the playback platform; Establish an initial scene, tool interface, drawing interface, learning interface and action playback function, and play the courseware file in the image display device and sound playback device according to the playback mode selected by the user in the user operation device and the established initial scene, tool interface, drawing interface, learning interface and action playback function.

12. The method according to claim 11, characterized in that The process of reading and parsing the courseware file includes: Read the length of the target file corresponding to the courseware file in json format; Read the json content in the target file corresponding to the courseware file; The three-dimensional model, audio content, video content and picture content are obtained according to the json content itself.

13. The method according to claim 12, characterized in that The process of establishing the initial scene, tool interface, drawing interface, learning interface and action playback function includes: Establishing an initial scene according to initial scene information describing content information and initial camera information; Establishing a tool interface according to the tool information describing the content information; Establish a drawing interface based on the drawing information describing the content information; A learning interface and an action playback function are established according to the process description information describing the content information.

14. The method according to claim 13, characterized in that The action playback function includes: A displacement action playback function for playing the displacement action of the three-dimensional model according to the displacement information of the process description information; An animation action playback function for playing the animation associated with the three-dimensional model according to the animation information of the process description information; A display image action playback function for playing pictures according to the display image information of the process description information; A display video action playback function for playing a video according to the display video information of the process description information; A modification mapping action playback function for playing the appearance changes of the three-dimensional model according to the modification mapping information of the process description information; A mobile camera action playback function is used to play the action of the camera space coordinates and orientation according to the mobile camera information of the process description information.

15. The method according to claim 14, characterized in that The description content information also includes test question information for recording test questions, the three-dimensional virtual simulation file also includes a test question file, and the method further includes: Displaying the assessment interface on the image display device; Obtaining option information selected by the user through the assessment interface; Comparing the option information with pre-stored answer information; An assessment result is obtained according to the comparison result, and the assessment result is displayed in the assessment interface.

16. A cross-platform three-dimensional virtual simulation courseware compilation system, characterized in that: The system includes a production module and a playback module; The production module is used to define the file format, target file location and identification method, create a user interaction interface, receive the three-dimensional virtual simulation file and the uploaded resource file generated by the user by operating the user interaction interface configuration, organize the three-dimensional virtual simulation file and the uploaded resource file according to the defined file format, target file location and identification method, and package the organized three-dimensional virtual simulation file and the uploaded resource file to generate a courseware file; The playing module is used to read and parse the courseware file, and play the courseware file according to the configuration of the playing platform and the playing mode selected by the user.

Images