Internet VR system based on finite state machine

Abstract

The present invention provides an Internet VR system based on a finite state machine, which belongs to the field of Internet technology. The system provides a software system that can clearly and quickly collect VR scene requirements, avoiding a series of risks caused by unclear requirements to scene design and implementation; decomposing scenes based on states, building scenes, and providing preview images in real time, and users can observe and zoom the preview images at different angles. The present invention provides an intuitive and effective communication method for developers and demand proposers, avoiding frequent rework caused by large differences between scene implementation and expected values.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to an Internet VR system based on a finite state machine. Background Art

[0002] Three.js is an open-source JavaScript framework based on WebGL. It serves as the foundation for many WebGL graphics engines and several browser-based game engines. It features a powerful, lightweight online editor. Three.js allows for 3D animations and interactivity through mouse, keyboard, and drag events. Adding 3D animation and interaction to a page can enhance the user experience. Three.js enables panoramic views, which can provide a more intuitive visual experience in the real estate and home improvement industries. In e-commerce, Three.js can be used to create 3D effects on products, allowing users to view products in 360 degrees, providing an enhanced shopping experience. It has a simple learning curve, hundreds of demos and examples, a rich tutorial library, and a strong community.

[0003] Until now, it's been impossible to clearly and quickly gather requirements for VR scenarios, leading to often unclear requirements at the outset of scenario development. Flaws in requirements work can significantly impact project outcomes, impacting user satisfaction and expectations. After gathering requirements, developing VR software from design to implementation takes a significant amount of time. Existing VR scenario development platforms don't offer intuitive scene previews during design, and implementation often deviates significantly from expectations, requiring frequent rework and wasting significant capital. Summary of the Invention

[0004] The object of the present invention is to provide an Internet VR system based on a finite state machine to solve at least one technical problem existing in the above background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In one aspect, the present invention provides an Internet VR system based on a finite state machine, comprising:

[0007] An identity determination module is used to determine the identity of the requester based on the request information input by the requester;

[0008] The scenario information input module is used to receive the projects, added scenarios and status information created by the scenario demander according to his / her own needs if the identity of the requester is a scenario demander;

[0009] The scene information preview module is used to generate a state diagram based on the created project, added scene and state information. The scene preview diagram of the state can be generated based on the corresponding node information of the state diagram. Based on the finite state machine, the scene is built with the state as the center and all the requirements of the scene are collected with the state as the center. The connection of multiple related states constitutes a dynamic scene. The nodes in the state diagram and the connections between nodes show the scene state information and the connection between different states.

[0010] The scene development module is used to edit the generated scene preview, change the position of objects in the scene, modify the scene, and export information of the current scene.

[0011] Preferably, the identity judgment module sends the input request information to the database for matching, and based on the matching result, it provides feedback to the user whether the login is successful and jumps to the corresponding interface. Each addition and deletion of a user account will cause changes in the data in the database. The id in the registration table in the database records the storage order of the user in the database and sets it as the primary key. The username is the user name and is set as a foreign key for subsequent search of other information under this account. The password is the account password, and the right is the role information corresponding to this user. Different roles have different permissions to use the system.

[0012] Preferably, the scene information input module includes a resource import unit for importing custom objects that will appear in the scene into the system.

[0013] Preferably, the scene information input module also includes a scene management unit for adding scenes and editing them. After the required materials are imported into this scene, the scene editing page can be entered, where the user can add different states to this scene according to needs, and then select a state and add objects according to needs and previously imported resource files, and set the behavior of the objects.

[0014] Preferably, in the scene information input module, the scene sequence table stores the scene names contained in the system. Since scenes are managed in units of projects, the id of the corresponding project to which the scene belongs is stored at the same time as the scene name for easy search; scenename is the scene name, and project_id is the id of the project to which the scene belongs. In order to avoid search errors caused by the same scene name, a unique id is assigned to each scene as the primary key.

[0015] Preferably, the user arranges objects and sets the behavior of objects in the required scene in the scene management unit, and the state diagram will be updated in real time with the user's behavioral operations, showing the state arrangement of the current scene; each node in the state diagram and the connection between nodes directly display the scene state information input by the user and the connection between different states. The state diagram and the scene preview diagram are bound to each other. Every time the user clicks on a state node in the state diagram, the preview area will generate a 3D preview of the corresponding state node.

[0016] Preferably, the scene information preview module updates the state diagram synchronously as the user adds state, object, and action information to the scene, and can generate a preview image of the corresponding state; the state diagram part uses the go.js framework to synchronize updates with the state input, and the preview image part uses the Babylon.js framework to generate.

[0017] Preferably, in the scene development module, if the scene preview effect does not match the requirements, the scene can be adjusted by continuing to add new states, objects or actions, or deleting the added scene information, and directly dragging the objects in the generated preview image to change their placement.

[0018] Preferably, if the scene preview needs to be adjusted, the editor is entered to adjust the object parameter details in the scene. When the scene is adjusted to meet the requirements, the currently constructed scene information is exported to complete the construction of the scene.

[0019] Preferably, the scene development module directly calls the data input by the scene information input module from the database to generate a state diagram and a scene preview diagram; the data changes involved are information about the position and rotation of the object, which will change the object position-related data in the database, and finally develop and derive the VR scene required by the user based on the data in the database.

[0020] The beneficial effects of the present invention are: it can clearly and quickly collect VR scene requirements, avoiding a series of risks in scene design and implementation due to unclear requirements; it decomposes the scene based on the state, builds the scene, and provides a preview image in real time, and the user can observe and zoom the preview image from different angles; it provides an intuitive and effective communication method for developers and demand proposers, avoiding frequent rework caused by large differences between the scene implementation and the expected value.

[0021] Additional advantages of the present invention will be more clearly given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 This is an architecture diagram of the finite state machine-based Internet VR system described in an embodiment of the present invention.

[0024] Figure 2 This is the database ER diagram described in an embodiment of the present invention.

[0025] Figure 3 This is a flow chart of the finite state machine-based Internet VR system according to an embodiment of the present invention.

[0026] Figure 4 This is a project management flowchart according to an embodiment of the present invention.

[0027] Figure 5 This is a diagram of a project management example according to an embodiment of the present invention.

[0028] Figure 6 This is a flowchart of the scenario input described in an embodiment of the present invention.

[0029] Figure 7 Input a Vue instance diagram for the scenario described in the embodiment of the present invention.

[0030] Figure 8 This is a scene preview flowchart according to an embodiment of the present invention.

[0031] Figure 9 Generate a function call flow chart for the state diagram described in the embodiment of the present invention.

[0032] Figure 10 This is a flowchart of the scenario generation function call described in an embodiment of the present invention.

[0033] Figure 11 This is a schematic diagram of a preview of a tooth extraction scenario according to an embodiment of the present invention.

[0034] Figure 12 This is a flowchart of the scenario development described in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0036] Those skilled in the art will understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0037] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0038] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0039] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise inconsistent.

[0040] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0041] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0042] Example 1

[0043] This embodiment 1 provides a Web VR system based on a finite state machine. A requirements collection module is designed and developed using technologies such as state machines, Vue, databases, and babylon.js, providing a platform that can clearly and quickly collect requirements for VR scenes, avoiding a series of risks brought about by unclear requirements to scene design and implementation. This system mainly uses technologies such as babylon.js to decompose scenes based on states, build scenes, and provide preview images in real time. Users can observe and zoom in on the preview images from different angles. Therefore, this system can effectively avoid a series of risks brought about by unclear requirements to scene design and implementation; it also provides developers and demand proposers with a more efficient communication method, avoiding frequent rework caused by large differences between the scene implementation and the expected value.

[0044] A finite state machine, also known as a state machine, consists of combinational logic circuits and state registers. It acts as a control center, transitioning between pre-set states based on control signals, completing specified operations and coordinating related signal actions. A state machine is not a machine, but rather a transition between states. For example, a person has three states: healthy, sick, and undergoing treatment. The conditions that trigger a state transition include being caught in the rain (a1), taking medicine (a2), getting an injection (a3), and resting (a4). The resulting state machine is: healthy (a1) -> sick; sick (a3) ​​-> healthy; sick (a2) -> undergoing treatment; undergoing treatment (a4) -> healthy, and so on. A state machine is a diagram in which states transition to their own or different states under different conditions. Related to finite state machines is the concept of finite state automata, but the two are not identical. The main difference is that finite state automata are a special type of finite state machine. Each state transition in a finite state machine generates an output, while finite state automata are finite state machines without outputs. Instead, they use terminal states, which are generally used for identification. Therefore, in this embodiment 1, the system builds the scene with the state as the center according to the definition of the finite state machine and its practical application, and collects all the requirements of the scene with the state as the center. The connection of multiple related states constitutes a dynamic scene.

[0045] Babylon.js is a complete JavaScript framework developed using the TypeScript language based on WebGL and JavaScript. It is used to build 3D games and experiences for HTML5, WebGL, WebVR, and Web Audio, and provides an easy-to-use interface for handling complex WebGL animations. For users, using the Babylon.js framework is easy. It includes all the necessary tools for creating and managing 3D objects, special effects, and sounds. As one of the most popular 3D game engines, Babylon.js is widely used by developers. As a 3D library, it has many commonly used objects and very easy-to-use tools built in. These features help developers implement common 3D functions in an efficient and accurate manner. Therefore, in the VR system of this embodiment 1, this framework is used to design and develop 3D preview images and VR preview images on the Web side.

[0046] Go.js is a JavaScript framework that makes it easy to draw interactive graphics and charts based on HTML5 browsers. It provides users with many advanced features such as drag-and-drop, automatic layout, templates, data binding, and models. Go.js adopts an object-oriented programming model. Drawing templates are represented by graphic objects. Data is stored in ordinary JavaScript objects as a data model, and then assigned to the properties of the graphic objects as a data binding model. Go.js also provides a large number of tool classes to represent our interactive behaviors. All we need to do is create graphic objects, build data models, set properties, bind data models, and use tool classes to add behaviors to create a variety of charts with rich interactive features. User interaction is achieved without switching servers or plug-ins, and it runs completely in the browser. It works with any HTML or JavaScript framework, even without any framework at all. Therefore, in the VR system of this embodiment 1, Go.js technology is used in the scene generation and preview module. In combination with other JavaScript frameworks, the state diagram is bound to the input of scene-related information to achieve real-time updating of the state diagram with input and interaction with the preview scene.

[0047] MySQL is a relational database management system and one of the most popular relational database management systems currently. It is closely integrated with mainstream programming languages ​​such as PHP and Java. MySQL has low complexity but still provides a wide range of database functions and options. In terms of web applications, MySQL is one of the best relational database management system application software. Relational databases use tables as data storage tools, storing data in different tables and categorizing them, rather than storing all data in a large warehouse. This improves the speed and flexibility of data storage and search. The huge storage capacity can easily handle large databases with tens of millions of records, and provides management tools for checking, managing, and optimizing database operations. MySQL has good compatibility with the popular web development language PHP. Therefore, in the VR system of this embodiment 1, the MySQL database is combined with PHP to implement data storage in the backend of the user information management module, as well as the storage of scene-related information in the project management module and the scene generation and preview module.

[0048] like Figure 1 As shown in Example 1, the finite state machine-based Web VR system is designed using a browser / server architecture (i.e., browser / server architecture). Users access the system through a web browser. Under this B / S architecture, users interact with the system through the browser, and the results are displayed in the browser. Only minimal transaction logic is implemented on the browser side, while the main business logic is implemented on the server side. This greatly simplifies the load on the client computer and reduces the cost of system maintenance and upgrades. Systems using a B / S architecture are easy to deploy and upgrade. Only the application on the server needs to be operated, and the client only needs to install a browser to interact with the system, which can significantly reduce system maintenance costs. The finite state machine-based Web VR system is state-centric, collecting user requirements, generating corresponding state previews (including state diagrams and 3D previews), generating corresponding VR scenes, and providing a basic modification interface. A web page is used to collect user requirements, and a state diagram and basic 3D preview are displayed on the web page for the user's scene settings.

[0049] The web page's primary focus is interface and interaction design, implemented using a combination of jQuery, Vue, Bootstrap, Go.js, and Babylon.js. Bootstrap's built-in global CSS settings, basic HTML element styles, and numerous components will be used to build and enhance the front-end interface. Vue's responsive data binding, combined view components, and two-way data binding will reduce the amount of internal data interaction required for front-end page interaction design. Two-way data binding primarily means that when the data changes, the view changes; and when the view changes, the data changes accordingly.

[0050] All interactions on the web page are implemented in the jQuery framework using JavaScript. Go.js is used to generate the scene state diagram, which is bound to the state input and interacts with the 3D preview generated by the Babylon.js framework.

[0051] In Example 1, the system deploys and runs the main program on a webpage, so all interactions within the system's webpage are designed using JavaScript. Leveraging the Vue componentization concept, user input information is categorized and placed in different Vue instances. For user input, most of the input occurs on the page and information exchange occurs between pages, primarily accomplished using the Vue framework. System-generated components such as state diagrams, preview images, and the final scene are handled using the go.js and Babylon.js libraries.

[0052] The server side will use Tomact and other servers with MySQL database to implement the system data storage, extraction and modification.

[0053] The database is used to store user account passwords and other related information, project information saved in the system, scene information, and the status, objects, and object actions in each scene. The above information is classified and stored in different tables. The system uses the database ER diagram as follows Figure 2 shown.

[0054] In this embodiment 1, the basic use process of this system is as follows: Figure 3 shown.

[0055] All users first log in to the system, where the system verifies their identity. Administrators proceed to user information management, while developers proceed to the scenario requirements collection step. The user creates a project, adds scenarios, and states based on their scenario needs. Alternatively, developers can input scenarios based on the requirements document provided by the user.

[0056] Then, a state diagram is generated according to the user's input requirements. Clicking the corresponding node in the state diagram can generate a scene preview of the state.

[0057] Each node in the state diagram and the connections between the nodes directly display the scene state information input by the user and the connections between different states. The state diagram is also bound to the preview image of the scene. Every time the user clicks on a state node in the state diagram, the preview area below will generate a preview image of the corresponding state node, and the user can observe the scene preview effects of different states. If the preview effect is significantly different from the requirements, the user or developer can continue to add new states, objects or actions, or delete the added scene information until the preview image generated by the current input roughly meets the requirements. Next, you can enter the full-screen mode of the web page to observe the scene in detail. If adjustments are required, enter the editor to adjust the parameter details such as the object properties in the scene. When the scene is adjusted to meet the requirements, the developer can export the currently built scene information to complete the construction of this scene.

[0058] Example 2

[0059] This embodiment 2 provides an Internet (Web) VR system based on a finite state machine. In this embodiment 2, the specific technical solution of the Internet (Web) VR system based on a finite state machine is introduced in blocks according to the usage process sequence.

[0060] The first step is to log in to the system. All users must log in to the system first. The system will determine the user's identity. If it is an administrator, it will enter the user information management step. If it is a demander or scenario developer, it will enter the scenario requirement collection step.

[0061] In this second embodiment, the database design of the system login identity determination module is as follows: When triggered, this part sends the input account, password, and other information to the database for matching. Based on the matching results, the user is fed back whether the login is successful and redirected to the corresponding interface. Each addition and deletion of a user account causes changes to the data in this table. id records the storage order of users in the database and is set as the primary key. username is the user name and is set as a foreign key for subsequent searches of other information under this account. password is the account password. right is the role information corresponding to this user. Different roles have different permissions for using the system. The specific table design is shown in Table 1.

[0062] Table 1 Register table

[0063]

[0064] The second is project management. Enter the project management interface, which presents the user with a list of all projects under the current account. All scenarios are managed in units of projects. Each project contains different scenarios involved in the project. Users can create and edit projects. After creating a project, users can click on the project details to enter the scenario management interface under the project. Click the delete button to delete the project. The operation process is as follows Figure 4 shown.

[0065] In this embodiment 2, when the user operates the project, the data involved in the vue instance is synchronized with the database to record the user's modifications and accurately display the existing data information on the interface. The classified information is stored in two vue instances, such as Figure 5 As shown:

[0066] The project_app instance controls the display, addition, and removal of the current project, using the following information: newProjectName (the name of the newly added project), currentProjectIndex (the index of the currently added project within the project collection), projects (an array of all projects), project_user (the username of the account holding the project), and project_description (the project description). The projects array stores each project's name, description, and id, as well as the scene collection it contains.

[0067] The choice_project_app instance is used to obtain the id of the project selected by the user for operation. The control information is projects (a collection array of all projects), scene_kinds_num (the number of scenes contained in the current project), and chosenprojectIndex (the sequence number of the project selected by the current user in the project array).

[0068] In this second embodiment, the database design of the system's project management module is as follows: For data storage in the database, each different account has its own project list. Therefore, this table stores the project name and also includes a field called username, which indicates which account the project belongs to. Since all account data is stored in a single database, to avoid inaccurate queries caused by the same project name across different accounts, a unique ID is assigned to each project and set as the table's primary key. The table also stores the project name, but users can choose not to enter a project description. The specific fields of the Project_list table are shown in Table 2:

[0069] Table 2 Project_list table

[0070]

[0071]

[0072] The Project Overview screen displays the projects under the current account. Clicking the Create Project button in the upper-right corner switches to the Project Information screen, where you can enter project-related information. Clicking OK successfully adds the project, while Cancel returns to the Project Overview screen. There are two options at the end of the project information list: To delete a project, click the Delete button. Clicking the Project Details button takes you to the Project Overview screen.

[0073] Then comes the scene input process. Figure 6 As shown, this part is divided into two parts: resource import and scene management. The resource import part can import some custom objects that will appear in the scene into the system. In the scene management part, users can add scenes and edit them. After the required materials are imported into this scene, they can enter the scene editing page. Here, users can add different states to this scene according to their needs, and then select a state and add objects according to their needs and the previously imported resource files, and set the behavior of the objects (state transfer). Users will follow the above basic process to set up the relevant scenes. This process shows the basic state-centered characteristics of the system, and implements the design concept of progressive layer by layer from project to scene, from scene to state, from state to object, and from object to action, that is, the user must select an option in the previous level before setting the relevant information of the next level.

[0074] like Figure 7 As shown, in this embodiment 2, by classifying and analyzing the information that may be involved in the scene input module, using the idea of ​​Vue componentization, different Vue instances are created to place this information respectively, involving a total of eleven instances, as shown in the design diagram. Figure 7 shown.

[0075] The alert_app instance is used to control the prompt box during irregular operations. The controlled information is isHidden (whether to hide) and alert_info (prompt information). The isHidden value is initially empty. When an incorrect operation triggers the listening event of this instance, the isHidden value will be modified according to the triggering event to control the display and hiding of the prompt box. The project_scene instance stores information about the scene, controls the display of all scenes in the current project, the addition of new scenes, and the removal of existing scenes. There are four types of information stored, namely newscene_name (name of the newly added scene), project_scenes (array of all current scenes), scene_index (the order of the currently selected scene in the scene array), and scene_description (scene description). In the project_scenes array, the scene information stored includes scene id, name (scene name), description, belong_project, and object_types (array of objects contained in the scene).

[0076] The scene_object instance stores information about objects imported into the scene. It controls the new_object_type (the name of the newly imported object type) and scene_objects (an array of all objects imported into the scene). Clicking Add Object in the interface triggers the addNewObjectKind method in the instance, which stores the object information in the new_object_type array and assigns the updated object array to the object_types array of the scene's objects, completing the update of the scene's object information.

[0077] The obj_detail instance is mainly used to store the paths of object models, images, and other material files that have been imported into the scene. The stored information includes img_src (the local storage path of the image / model file) and position (the built-in position of the object model).

[0078] The select_scene instance is used to obtain the scene information selected by the user. The stored information includes scenes (a collection array of all scenes contained in the project), obj_type_num (the number of object types contained in the selected scene), and selectscene_index (the sequence number of the scene selected by the user in the scene array).

[0079] The state_app instance responds to user actions such as adding or deleting states. It stores information such as newstate_name (the name of the newly added state), state_index (the index of the newly added state in the state collection array), states (the state collection array for the current scenario), and nextstate_index (the index of the next newly added state in the state collection array). The event listener in this instance monitors changes in the states array. Once this array changes, the state chart generation function creat_state_chart() is called to add the newly added state node based on all current state information and update the state chart in real time.

[0080] The select_obj instance is used to identify and determine the object that the user currently wants to add to the state. The information stored is objecttype_index (the serial number of the object currently selected by the user in the object collection array) and object_types (the collection array of all optional item types in the current state).

[0081] The state_object instance is used to respond to the addition and removal of objects and store the objects in the current state. It controls the following information: newObjectName (the name of the newly added object), currentObjectIndex (the object's index in the object collection array for this state), objects (the collection array of all objects in this state), and nextObjectId (the index of the next object stored in the array). The event listener in this instance monitors changes in the objects array. Once this array changes, the state chart generation function creat_state_chart() is called to add the newly added object node based on all current state information, updating the state chart in real time.

[0082] The target_state instance is used to indicate and lock the selected target state. The control information is targetstate_index (the serial number of the target state in the state collection array) and target_states (the currently selectable state collection array, which copies the states state collection array stored in the state_app instance).

[0083] The obj_action instance is used to respond to user requests for the creation and deletion of object actions and store information about already created actions. This instance stores information such as action_name (the name of the newly added action), actions (an array storing all actions for the currently selected object), and nextaction_index (the index of the next action added within the actions array). The event listener in this instance monitors changes in the actions array. Whenever this array changes, the statechart generation function, creat_state_chart(), is called to connect the state nodes involved in the action based on all current state information and update the statechart in real time.

[0084] The position_app instance is used to store the spatial information of objects in the state scene, covering information such as the object's coordinates and rotation angle. The stored information includes position (an array of information collection of the object in the scene space), this array contains location (the coordinates of the object in the spatial coordinate axis), and rotation (the rotation angle of the object). In the initial state, all spatial information is built-in by the system.

[0085] In this second embodiment, the database design for the scene information input module is as follows: The scene_list table stores the names of the scenes included in the system. Since scenes are managed by project, the ID of the corresponding project to which the scene belongs is stored along with the scene name for easy search. "scenename" represents the scene name, and "project_id" represents the ID of the project to which the scene belongs. To avoid search errors caused by duplicate scene names, each scene is assigned a unique ID as the primary key. Users do not need to enter a scene description when adding a scene. The specific data fields are shown in Table 3.

[0086] Table 3 Scene_list table

[0087]

[0088] The scene information input module of this system includes a resource import section. When creating a scene later, users can name the imported objects and add them to the state. The Scene_objs_list table stores the information of the resources (i.e., objects) imported into the scene. The table stores the object's name (obj_name), the scene ID (obj_sceneid), and the object's resource path (obj_file) for easy retrieval when needed. The Scene_objs_list table structure is shown in Table 4:

[0089] Table 4 Scene_objs_list table

[0090]

[0091] The Scene_states_list table stores information about states that have been created in the system. Since the same account can have different projects and scenes, the table stores the state name (state_name) and the scene ID (state_sceneid) to which the state belongs to accurately query the states included in the scene. The Scene_states_list table structure is shown in Table 5:

[0092] Table 5 Scene_states_list table

[0093]

[0094]

[0095] The State_objs_list table stores information about objects added to a state. It is the table that stores the most information and is also the most important. Since users need to name objects when adding them to a state during scene creation, the object names at this time are different from the object names when the resources are imported. Therefore, the table stores two data: name and obj_name. name is the name of the object in the state, and obj_name is the name of the object when the resources are imported, which is also the object type. The table also stores the target state (target), action name (action_name), state (obj_state), and spatial position information corresponding to the object. The State_objs_list table structure is shown in Table 6:

[0096] Table 6 State_objs_list table

[0097]

[0098] In this embodiment 2, the scene information input module has a scene overview interface. This interface displays all scenes in the project selected by the user. The user can create a scene according to his or her own needs, and then click on the material upload pop-up window to pop up the material upload window. Here, the resource files of the objects required in the scene are imported into the scene. After the resources are imported, you can click on the scene edit to enter the scene editing interface.

[0099] In the pop-up interface, there is also an information input section for scene editing (also called scene generation interface) based on the prototype design. Users will enter almost all the information needed to generate the scene in this interface, so it is the most important part of this system. Objects belong to states, and actions belong to objects. Users must first add and select a state before they can proceed with the next addition operation. This well reflects the state-centric design concept of this system. Users first need to add different states to the scene, and only after selecting an added state can the user add the objects involved in this state. Similarly, only after selecting an object in a certain state can the action information of this object be added.

[0100] Next is the scene preview module process. After completing the above project management, scene input and other operations, you can start to generate a state diagram based on the user's input requirements, such as Figure 8 As shown, clicking the corresponding node in the state diagram can generate a scene preview diagram of the state.

[0101] The scene information preview module is designed to provide real-time feedback on user input of scene status, objects, and scene previews. The core components are the state graph that can be dynamically generated based on user input and the 3D scene graph bound to the state graph.

[0102] In the scene management module, users arrange objects and set the behavior of objects in the required scene. The state diagram will be updated in real time as the user's behavior and operations are performed, showing the state arrangement of the current scene. The various nodes in the state diagram and the connections between nodes directly display the scene state information entered by the user and the connections between different states. The state diagram is also bound to the scene preview image. Every time the user clicks on a state node in the state diagram, the preview area below will generate a 3D preview image of the corresponding state node, and the user can have an intuitive preview of the scene and the objects in it. The user can observe the scene preview effects in different states. If the preview effect is significantly different from the requirements, the user or developer can continue to add new states, objects or actions, or delete the added scene information, until the preview image generated by the current input roughly meets the requirements.

[0103] The scene preview component updates the state diagram as the user adds states, objects, actions, and other information to the scene. Clicking on the state diagram generates a preview of the corresponding state. The state diagram uses the go.js framework to update synchronously with state input, while the preview is generated using the Babylon.js framework.

[0104] The state diagram above the preview section leverages Vue's responsive data binding, combined view components, and MVVM (Model-View-ViewModel, a two-way data binding model)—three features, along with the go.js framework's flowcharting capabilities—to enable real-time updates based on user scenarios. Each time a user adds a state, a state node is generated in the state diagram. Each time an object is added to a state, an object node connected to that state is generated in the state diagram. If the user sets an action for an object, the two related states in the state diagram are connected. Generating a state diagram requires simply adjusting the state diagram instance in the go.js framework. Deleting and creating nodes in the state diagram is accomplished by the system monitoring the user-entered data and the currently selected Vue instance.

[0105] In the process of generating the state diagram, the requirements for generating the state diagram are decomposed and the Figure 9 The generated function call flow is shown.

[0106] At the end of the file containing the Vue instance code, six functions are designed: get_point, get_obj_rel, get_all_points, get_location, get_action_rel, and creat_state_chart. The get_point function stores each state and object information input into an array named point, and then stores it into the space_array array as a node in the state diagram. The get_obj_rel function obtains the action information of the object as the relationship between the two state nodes in the state diagram. After the user enters a scene of the project, the preview area is blank at first. As the user adds states, objects, and actions, the corresponding Vue instance monitors the changes in the corresponding array and calls creat_state _chart function, call get_obj_rel() and get_all_points() in the create_state_chart function in sequence, and then call other functions inside the two functions. Through the mutual calls between them, all the scene information including state, objects, actions, etc. of the current input is obtained, and stored in an array called space_array. Then, this array is converted into a JSON string through the JSON.stringify method and submitted to the background. Finally, the node and relationship arrays obtained by the previous function are used to set the flowchart data result_json, and the load function in the flowchart drawing code file is called. The state diagram instance in the modified go.js framework can be used to generate a state diagram of the user's current input information based on the transmitted flowchart data.

[0107] like Figure 10 The following figure shows the function call relationship involved in the scene preview image generation process. Before the user clicks a node in the state diagram, the scene preview image area is blank. When the user clicks a node, the system will identify the node clicked by the user and first call initiateScene to create an empty scene for the current state. The preview image area is still blank at this time. Then, createAsset is called to load basic resources for this empty scene (such as built-in system object models, cameras, lights, and scene materials uploaded by the user). Next, the createScene function is called based on the state node selected by the user to create the basic objects that should appear in the scene graph of the corresponding state (such as classroom-style floors, ambient colors, ceilings, etc. in a classroom scene). Finally, the system calls analyzePoints to analyze the node information in the state diagram, analyze the clicked state and the objects connected to this state, and calls loadScene to load these object models. If a preview image has been generated before clicking the new state node, purifyScene and reloadScene are called to clear the objects in the current scene and then reload the preview image of the switched state.

[0108] In this embodiment 2, the database design of the scene information preview module is as follows: almost all the data involved in this part are input by the user in the scene input module, and the information input by the user in the scene input part is directly called from the database to generate the state diagram and scene preview diagram. After the user enters the information, the input information will be normalized by the system and bound to the state diagram, becoming the information of each node in the state diagram, so that the state diagram can be updated in real time. Figure 11 As shown in the figure, the upper part is the state diagram display part, and the lower part is the preview display part. The user can click a state node in the upper state diagram (such as "State 1: Periosteum Separation") to display or switch the scene state shown in the preview diagram. Similarly, the user can click an object node in the state diagram (such as "Hand: Hand 1") to focus the camera on the selected object.

[0109] The last part is the scenario development module process. Figure 12As shown, the main functions of this part are editing the generated scene, dragging objects in the scene to change their positions, using the editor to modify the scene, and exporting information of the current scene. If the preview effect does not meet the requirements, the user or developer can continue to add new states, objects or actions, or delete the added scene information, and directly drag the objects in the generated preview image to change their placement to adjust the scene. You can also enter the full-screen mode of the web page to observe the scene in detail. If adjustments are required, enter the editor to adjust the parameter details such as the object properties in the scene. When the scene is adjusted to meet the requirements, the developer can export the currently built scene information to complete the construction of this scene.

[0110] This module provides developers with a tool for developing and modifying user-entered scene requirements. By clicking the full-screen mode button, developers can enter the full-screen view of the webpage. They can use the mouse to change the viewing angle or press WASD to move the camera. Using the Ctrl+Alt keys on the keyboard brings up the babylon.js scene editor and begin editing the current scene.

[0111] In the present embodiment 2, the scene editor of babylon.js is used. In the display interface, the right column shows all the entities in the scene, and the left column lists the various parameters of the entity clicked by the user in the figure or on the right. The user can click the mouse to select the model in the scene, or click the model name in the list on the right, and then click the move, rotate, and zoom icons above the scene to modify the detailed properties of the object and perform certain debugging. After clicking the object, the name, material, state, spatial position and other information of the clicked object will be displayed in the left column, and the various parameters of the object can be modified directly in the input box. Each change of the various parameters of the object will also be transmitted to the corresponding Vue instance in real time, and then the information in the two-way binding editor and the information in the Vue instance are also conducive to saving and exporting the scene information in the page after the construction is completed.

[0112] In this embodiment, the database design for the scene development module is as follows: Almost all of the data involved in this section is user input into the scene input module. This information is directly retrieved from the database to generate the state diagram and scene preview. The data changes involved are object position and rotation information, which are modified in the State_objs_list table in the database. Finally, the development module exports the user's desired VR scene based on the database data.

[0113] In summary, the finite state machine-based internet VR system provided by the embodiments of the present invention designs and implements a scene input function. This function is divided into resource import and scene management. Resource import primarily involves the user uploading the 3D models and images required for the scene. In scene management, users can create different projects based on their needs, add different scenes that may be involved to the project, add different states and the objects involved in each state within a fixed scene, and set object state transitions (i.e., the desired behaviors within that scene). Therefore, this system can comprehensively and clearly collect requirements, laying a good foundation for subsequent VR scene development. A scene generation preview function has been designed and developed. Developers can generate a state preview diagram based on the information in the scene input module. The state diagram displays all states in the scene, the objects involved in each state, and the connecting actions between states. Users can further adjust the states and objects involved in the scene as needed. Clicking the corresponding state node in the state diagram generates a 3D preview of that state, which users can observe and zoom in and out from different angles. This function will help developers and requesters reduce communication costs and rework frequency, thereby saving a significant amount of human resources.

[0114] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0116] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0118] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.

Claims

1. An Internet VR system based on a finite state machine, characterized in that: include: An identity determination module, used to determine the identity of the requester based on the request information input by the requester; The scene information input module is used to receive the project, added scene and status information created by the scene demander according to his own needs if the identity of the requester is a scene demander; The scene information preview module is used to generate a state diagram according to the created project, added scene and state information, and generate a scene preview diagram of the state according to the corresponding node information of the state diagram; based on the finite state machine, the scene is built with the state as the center, and all the needs of the scene are collected with the state as the center, and multiple related states are connected to form a dynamic scene; each node in the state diagram and the connection between nodes show the scene state information and the connection between different states; among them, the six functions get_point, get_obj_rel, get_all_points, get_location, get_action_rel, and creat_state_chart are designed. The get_point function stores each input state and object information into an array named point, and then stores it into the space_array array as a node in the subsequent state diagram. The get_obj_rel function obtains the action information of the object. For the relationship between two state nodes in the state diagram, after the user enters a scene of the project, the preview area is blank at first. As the user adds states, objects, and actions, the corresponding Vue instance monitors the changes in the corresponding array and calls the creat_state_chart function. In the create_state_chart function, the get_obj_rel and get_all_points functions are called in sequence, and other functions are called inside to obtain all the scene information currently input, and store it in an array called space_array. Then, this array is converted into a JSON string through the JSON.stringify method and submitted to the background. Finally, the node and relationship arrays obtained by the previous function are used to set the flowchart data result_json, call the load function in the flowchart drawing code file, and use the state diagram instance in the modified go.js framework to generate a state diagram of the user's current input information based on the transmitted flowchart data; The scene development module is used to edit the generated scene preview image, change the position of objects in the scene, modify the scene, and export the information of the current scene.

2. The finite state machine-based Internet VR system according to claim 1, characterized in that: The identity judgment module sends the input request information to the database for matching, and based on the matching results, it provides feedback to the user whether the login is successful and jumps to the corresponding interface. Each addition and deletion of a user account will cause changes in the data in the database. The id in the registration table in the database records the storage order of the user in the database and sets it as the primary key. The username is the user name and is set as a foreign key for subsequent searches of other information under this account. Password is the account password, and right is the role information corresponding to this user. Different roles have different permissions to use the system.

3. The finite state machine-based Internet VR system according to claim 1, characterized in that: The scene information input module includes a resource import unit, which is used to import custom objects that will appear in the scene into the system.

4. The finite state machine-based Internet VR system according to claim 3, characterized in that: The scene information input module also includes a scene management unit, which is used to add scenes and edit them. After the required materials are imported into this scene, the scene editing page can be entered, where the user can add different states to this scene according to needs, and then select a state and add objects according to needs and previously imported resource files, and set the behavior of the objects.

5. The finite state machine-based Internet VR system according to claim 4, characterized in that: In the scene information input module, the scene sequence table stores the scene names contained in the system. Since scenes are managed in projects, the id of the corresponding project to which the scene belongs is stored together with the scene name for easy search. scenename is the scene name, and project_id is the id of the project to which the scene belongs. In order to avoid search errors caused by the same scene name, a unique id is assigned to each scene as the primary key.

6. The finite state machine-based Internet VR system according to claim 5, characterized in that: The user arranges objects in the required scene and sets the behavior of objects in the scene management unit. The state diagram will be updated in real time with the user's behavior operations, showing the state arrangement of the current scene; each node in the state diagram and the connection between nodes directly display the scene state information input by the user and the connection between different states. The state diagram and the scene preview diagram are bound to each other. Every time the user clicks on a state node in the state diagram, the preview area will generate a 3D preview of the corresponding state node.

7. The finite state machine-based Internet VR system according to claim 6, characterized in that: The scene information preview module updates the state diagram synchronously as the user adds state, object, and action information to the scene, and can generate a preview of the corresponding state. The state diagram part uses the go.js framework to update synchronously with the state input, and the preview part uses the Babylon.js framework to generate.

8. The finite state machine-based Internet VR system according to claim 1, characterized in that: In the scene development module, if the scene preview effect does not meet the requirements, you can adjust the scene by continuing to add new states, objects or actions, or deleting the added scene information, and directly dragging the objects in the generated preview image to change their placement.

9. The finite state machine-based Internet VR system according to claim 8, characterized in that: If you need to adjust the scene preview, enter the editor to adjust the object parameter details in the scene. When the scene is adjusted to meet the requirements, export the currently built scene information to complete the construction of this scene.

10. The finite state machine-based Internet VR system according to claim 9, characterized in that: The scene development module directly calls the data input by the scene information input module from the database to generate a state diagram and a scene preview diagram; the data changes involved are the information on the position and rotation of the object, which will change the object position-related data in the database, and finally develop and export the VR scene required by the user based on the data in the database.

Images