Method and system for synchronizing data

Abstract

Methods and systems for synchronizing interactions in an immersive environment and a non-immersive environment of a software application are described. A pairing is formed between an interactive element in the immersive environment and an interactive element in the non-immersive environment. In response to an interaction with a first interactive element of the paired interactive elements, a message is communicated from the environment of the first interactive element to the environment of the second interactive element of the pairing, comprising a pairing identifier and state data associated with the first interactive element. When the message is received in the environment of the second interactive element, the second interactive element is identified based on the pairing identifier, and the state of the second interactive element is modified dynamically based on the state data.

Description

METHOD AND SYSTEM FOR SYNCHRONIZING DATATECHNICAL FIELD

[0001] The present disclosure relates to methods and systems for synchronizing data between different application environments, and more particularly to real-time synchronization between immersive environments and non-immersive environments.BACKGROUND

[0002] In recent decades, immersive technologies have become prevalent in both consumer technology’ markets and commercial settings. Immersive technologies combine hardware and software elements to place users in immersive environments. In virtual reality (VR) systems, a user is placed in a fully simulated three-dimensional digital environment. In mixed-reality systems, a computer-generated virtual world is superimposed on a real world, with both virtual and physical objects co-existing in the mixed reality environment, and virtual objects responding to changes in the physical world. In augmented reality (AR) systems, virtual objects are overlaid on the real -world. In augmented reality systems, there is less interaction between the virtual and the real worlds than mixed reality' systems.

[0003] While immersive technology has been utilized across various fields for many years, recent advancements have significantly broadened its industrial application. For example, immersive systems are now being used to support computer-aided design (CAD) and manufacturing (CAM), advanced simulation and testing, production line management, factory planning, and many other similar applications. Immersive environments provide enhanced spatial understanding, intuitive three-dimensional manipulation, and collaborative review capabilities that are particularly valuable for design visualization and evaluation.

[0004] Some systems have capabilities that allow simultaneous use of a software application in both immersive and non-immersive environments. In the context of modelling systems with these capabilities, the non-immersive environment may remain the primary source of geometric and non-geometric object creation and modification, while the immersive environment provides a representation of the same data in VR or AR for the purpose of visualization. The data models and objects in the non-immersive environment may differ significantly from their representations in the immersive environment. Moreover, such systems are generally unidirectional: while the system may support data transfer from the non-immersive environment to the immersive environment, there may be limited or no capabilities that enable creation and modification of data in the non-immersive environmentdirectly from the immersive environment. This limits the user and interrupts their workflow as every time the user wants, for example, to modify a CAD model, they are forced to return to the non-immersive environment to make the modification. Additionally, the inability to use the immersive environment to invoke and control tools available in the non-immersive environment prevents users from taking full advantage of the spatial interaction capabilities that are available in the immersive environment.

[0005] It would therefore be desirable to have a system that provides full bidirectionality between immersive and non-immersive environments.SUMMARY

[0006] It is an object of the present disclosure to provide a system and method that facilitates real-time bidirectional data transfer and synchronization between immersive and non-immersive environments.

[0007] The foregoing and other objects are achieved by the features of the independent claims. Further implementation forms are apparent from the dependent claims, the description, and the figures.

[0008] According to a first aspect, a computer-implemented method of synchronizing an interaction in an immersive environment and a non-immersive environment of a software application is provided. The method comprises: forming a pairing between an interactive element in the immersive environment and an interactive element in the non-immersive environment, detecting an interaction with a first one of the paired interactive elements in either the immersive environment or non-immersive environment, and in real-time, in response to detecting the interaction: communicating a message from the environment of the first interactive element to the environment of the second interactive element of the pairing, wherein the message comprises a pairing identifier and state data associated with the first interactive element, receiving the message in the environment of the second interactive element, identifying the second interactive element based on the pairing identifier, and dynamically modifying the state of the second interactive element based on the state data.

[0009] In a first implementation form of the first aspect, the software application is a modelling application.

[0010] In a second implementation form, a model is display ed in the non-immersive environment and a corresponding model is displayed in the immersive environment.

[0011] In a third implementation form, detecting an interaction comprises detecting a change of state of the first interactive element resulting from a model modification initiated in the environment of the first interactive element.

[0012] In a fourth implementation form, the state data comprises updated parameter values for a set of parameters shared by the first interactive element and the second interactive element, in response to the model modification.

[0013] In a fifth implementation form, dynamically modifying the state of the second interactive element comprises updating the corresponding parameters of the second interactive element based on the updated parameter values, resulting in a corresponding modification to the model in the environment of the second interactive element.

[0014] In a sixth implementation form, the method further comprises: detecting an interaction with the second interactive element, and, in real-time, in response to detecting the interaction: communicating a message from the environment of the second interactive element to the environment of the first interactive element, wherein the message comprises the pairing identifier and state data associated with the second interactive element; receiving the message in the environment of the first interactive element, identifying the first interactive element based on the pairing identifier, and dynamically modify ing the state of the first interactive element based on the state data.

[0015] In a seventh implementation form, the synchronization is bidirectional such that interactions initiated in either the immersive environment or the non-immersive environment are applied in the other environment in real-time.

[0016] In an eighth implementation form, the method comprises, for each interactive element, storing the current state data of the interactive element and the pairing identifier in a data structure associated with the interactive element, and updating the data structure based on data received from the environment of the other interactive element.

[0017] In a ninth implementation form, the pairing is maintained in each environment in a list of paired interactive elements that stores the pairing identifier and a reference to the corresponding interactive element.

[0018] In a tenth implementation form, the message is a JavaScript Object Notation (JSON) message, and wherein the pairing identifier and state data are appended to the JSON message.

[0019] In an eleventh implementation form, the method comprises instantiating an interactive element in response to an initial user interaction in a first one of the immersive environment and the non-immersive environment.

[0020] In a twelfth implementation form, the method further comprises instantiating an interactive element in the second one of the immersive environment and the non-immersive environment, in response to an initial message received from the first environment.

[0021] In a thirteenth implementation form, the model modification comprises one or more operations including, but not limited to: translate, rotate, scale, extrude, fillet, chamfer, create feature, delete feature, change constraint, change visibility, mirror, pattern, loft, sweep, revolve, shell, Boolean, split, edit sketch, and change material.

[0022] These and other aspects of the disclosure are apparent from the embodiments described below.BRIEF DESCRIPTION OF THE DRAWINGS

[0023] For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0024] Figure 1 shows a system providing immersive and non-immersive environments, according to an example.

[0025] Figure 2A shows anon-immersive environment of a modelling system, according to an example.

[0026] Figure 2B shows an immersive environment of a modelling system, according to an example.

[0027] Figure 3 shows a flowchart of a method for synchronizing interactions in an immersive environment and a non-immersive environment, according to an example.

[0028] Figure 4 is a diagram showing data structures for synchronizing interactions in an immersive and anon-immersive environment, according to an example.

[0029] Figure 5 shows an example of a data processing system in which embodiments of the present disclosure may be implemented.DETAILED DESCRIPTION

[0030] Example embodiments are described below in sufficient detail to enable those of ordinary skill in the art to embody and implement the systems and processes herein described. It is important to understand that embodiments can be provided in many alternate forms and should not be construed as limited to the examples set forth herein.

[0031] Accordingly, while embodiments can be modified in various ways and take on various alternative forms, specific embodiments thereof are shown in the drawings and described in detail below as examples. There is no intent to limit embodiments to the particular forms disclosed. On the contrary, all modifications, equivalents, and alternatives falling within the scope of the appended claims should be included. Elements of the example embodiments are consistently denoted by the same reference numerals throughout the drawings and detailed description where appropriate.

[0032] The terminology used herein to describe embodiments is not intended to limit the scope. Elements referred to in the singular can number one or more unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and / or “including,” when used herein, specify the presence of stated features, items, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, items, steps, operations, elements, components, and / or groups thereof.

[0033] Unless otherwise defined, all terms including technical and scientific terms used herein are to be interpreted as is customary' in the art. It will be further understood that terms in common usage should also be interpreted as is customary’ in the relevant art and not in an idealized or overly formal sense unless expressly so defined herein.

[0034] Methods and systems described herein are implemented in or in conjunction with a modelling system. Herein a modelling system may refer to a Computer-Aided Design (CAD) system, a Computer-Aided Engineering (CAE) system, a Computer-Aided Manufacturing (CAM) system, or similar. Modelling systems enable the creation, modification and analysis of a design of an object for manufacture. Modelling software is used across many industries, including architecture, engineering, manufacturing, and product design. Modem modelling systems enable the creation of highly detailed two and three dimensional models and provide a vast array of modelling and design tools to enable designers to efficiently modify’ models yvithout having to reconfigure an entire design by hand. Modelling systems can integrate yvith other software tools, such as simulation software, product lifecycle management (PLM) systems, and other systems.

[0035] Figure 1 shows a system 100 for interacting with a software application in both an immersive environment and a non-immersive environment, on which embodiments of the disclosure may be implemented. In examples described herein, the software application maybe a modelling system such as a CAD system. However, the methods described are applicable to other software applications and systems including, but not limited to, simulation systems and PLM systems. Herein an immersive environment may refer to a computergenerated virtual reality environment, a mixed-reality environment, an augmented reality environment, or similar. A user 110 operates the system 100 and interacts with both the immersive and non-immersive environments. The user 110 wears an immersive headset 120 that provides access to the immersive environment. The headset 120 uses stereoscopic technology to give a sense of depth in a field of view 130 of the user 110, and sensors, such as accelerometers and gyroscopes, which track the user’s head movements in real time.

[0036] In the example shown in Figure 1, a geometric object 140 is displayed in the field of view 130 of the immersive environment. The user 110 manipulates objects in the immersive environment using a controller 150. The controller 150 is a user input device that is communicatively coupled with the headset 120. The controller 150 may comprise one or more handheld or wearable devices and may provide various features to provide user input including, but not limited to: buttons, control pads, touch sensitive surfaces and the like. The controller may include sensors of various kinds including, but not, limited to: IR sensors, motion detection sensors, inertial sensors and the like. The controller 150 enables the user 110 to control tools in the immersive environment such as grab tools, selection tools, and other object manipulation tools.

[0037] The system 100 further comprises a computing system 160. The computing system 160 implements the non-immersive environment. The computing system 160 may comprise a display, data processors, memory, user input devices, and various interfaces including a communication interface to enable data to be transmitted to the headset 120, which may also include similar communication interfaces. Although the computing system 160 in Figure 1 is a desktop computer, any suitable computing system such as a laptop, handheld device, tablet, mobile device or similar may be used in conjunction with the methods described herein. In some cases, the non-immersive environment may be hosted on a remote server or cloud computing platform and rendered locally on a computing device. The display of the computing system 160 may display a user interface 170 that provides the non-immersive environment of the software application to the user 110.

[0038] The user 110 may interact with the user interface 170 of the non-immersive environment through user input devices such as a mouse and keyboard 180. The user input devices 180 enable the user 110 to perform actions in the non-immersive environment. In theexample shown in Figure 1, the user interface 170 presents a geometric object 190 that corresponds to the geometric object 140 visible in the immersive environment.

[0039] The system 100 enables the user 110 to work with the same software application data through either the immersive interface provided by the headset 120 and controller 140, or through the traditional interface provided by the computing system 160, user interface 170, and user input devices 180.

[0040] The methods described herein provide the ability to perform operations from either the immersive environment on data in the non-immersive environment or vice-versa as provided in the system 100. This is realized by extending bidirectional communication channels between the immersive environment and non-immersive environment to synchronize states of corresponding interactive elements.

[0041] Initially , interactive elements that drive any authoring capabilities in the non-immersive environment are identified. In the immersive environment, corresponding interactive elements are identified or created if a suitable candidate for a mirrored interactive element does not exist. The non-immersive environment triggers the creation of a corresponding interactive element in the immersive environment. Although the appearance and attributes of these interactive elements in the two environments may differ prominently, there are some common key attributes in both environments. These key attributes primarily decide the state of the interactive element and overall system. The real time state synchronization is achieved by communicating these common attributes between environments. It is assumed that at any particular moment, only one of the two environments is driving the state of its own interactive elements, and that the interactive element from the alternate environment is merely responding to the change in state.

[0042] The behavior of each interactive element is extended to have the ability to both broadcast to, and receive messages from, the other environment. When such an interactive element finishes its update in the environment that is driving the authoring change, a message describing the state of the interactive element is broadcasted to the driven environment. In that driven environment, the interactive element state is updated, and a global update is triggered.

[0043] A method is provided to ensure that each message is delivered to the correct mirrored interactive element in the other environment. Lists of paired interactive elements are maintained in both the environments. A unique pair identifier is assigned and kept alongside the mirrored interactive elements in both applications. Every transiting message carries thisunique pair identifier, and the book-keeping logic checks for this identifier to map each interactive element in its respective environment.

[0044] Figure 2A and Figure 2B illustrate the application of the methods described herein in non-immersive and immersive environments of a modelling system. Figures 2A and 2B demonstrate bidirectional synchronization between the environments when applied to a modelling operation that moves an object from its original position 100mm in the forward X-direction.

[0045] Figure 2A shows a non-immersive environment 200, which may be displayed to a user in the user interface 170. An initial position of a geometric entity 210, in this case a rectangular block identified in the non-immersive environment as NativeBlock, is represented by a dashed rectangular outline. A coordinate system indicator 220 is displayed in the lower left comer of the non-immersive environment 200, showing perpendicular axes that define the spatial reference frame for geometric operations.

[0046] The movement of the geometric entity 210 is controlled initially via a dialog box 230. The dialog box 230 is positioned in the upper left portion of the user interface. The dialog box 230 specifies "Entity To Move: NativeBlock," which identifies the geometric entity 210 to be moved, and "Distance to move: 100mm," which defines the distance parameter for the geometric transformation. The dialog box 230 further comprises an OK button to execute the Move Entity command.

[0047] When the Move Entity command is executed, the non-immersive environment 200 activates an interactive element 240 that changes the position of geometric entity 210 from its initial position 100mm in the forward A- axis direction to a subsequent position represented by the solid rectangular outline. The state data associated with the interactive element 240 includes length parameters that control the positioning of geometric objects such as the geometric entity 210.

[0048] Figure 2B illustrates an immersive environment 250 displaying a corresponding geometric entity 260 as may be displayed in the headset 120 of the user 110 in Figure 1. An initial position of the geometric entity 260 in the immersive environment 250 is represented by a dashed rectangular outline, corresponding to the initial position 210 in the non-immersive environment 200. A coordinate system indicator 270 is displayed in the lower left comer of the immersive environment 250. representing the spatial dimensions and reference frame of the immersive environment 250. A controller 280 is shown in the lower right portionof the immersive environment 250, depicted as a handheld input device for user interaction within the immersive environment 250.

[0049] As soon as the interactive element 240 is activated in the non-immersive environment 200 a corresponding interactive element 290 is created in the immersive environment 250 and paired with the interactive element 240. Both interactive elements 240, 290 enable change of the data model in the non-immersive environment 200. Tn this example, the controlling parameter is the length of the interactive elements 240, 290. Manipulation of the interactive element 290 with the controller 280 modifies the state data of the interactive element 290. Once the positioning update is completed in the immersive environment 250. a message is communicated to the non-immersive environment 200 to modify the state of interactive element 240 with the notified new length. Additionally, after updating state of the interactive element 240, change is propagated to the global data model to move the geometric entity 210, based on new length of the interactive element 240.

[0050] Similarly, when the controlling parameter for interactive element 240 is modified using, for example, the mouse cursor in the non-immersive environment 200, the change in length of the parameter is communicated to the immersive environment 250 and the interactive element 290 updates itself to bring it into synchronization with the non-immersive environment 200. The pairing between interactive elements 240, 290 therefore enables users to perform geometric manipulations bidirectionally via user input devices in either one of the environments 200, 250.

[0051] Figure 3 shows a flow diagram of a method 300 of synchronizing an interaction in an immersive environment and a non-immersive environment of a software application, according to an example. The method 300 provides a sequential flow of operations that enables real-time bidirectional synchronization between paired interactive elements across immersive and non-immersive environments of a softw are application such as a modelling system, as shown in Figures 2A and 2B. The method 300 may be implemented in conjunction with the other methods and systems described herein. In particular, the method 300 may be implemented on the system 100 shown in Figure 1.

[0052] At step 310, a pairing is formed between an interactive element in the immersive environment and an interactive element in the non-immersive environment. According to an example, an interactive element may be instantiated in response to an initial user interaction in the non-immersive environment. Following instantiation in the non-immersive environment, a corresponding interactive element may be instantiated in the immersiveenvironment and a pairing may be formed between the interactive elements. The pairing establishes correspondence between elements in different environments by assigning unique pairing identifiers to each pair of interactive elements. The different pairings may be maintained in each environment as a list or other data structure that stores the pairing identifiers and references to the corresponding interactive elements. This data structure enables the system to track and manage multiple paired interactive elements simultaneously across both environments.

[0053] At step 320, an interaction with a first one of the paired interactive elements is detected in either the immersive environment or non-immersive environment. In examples, the detection of an interaction may comprise a detection of a change of state of the first interactive element resulting from modifications initiated in the environment of the first interactive element. In the case of a modelling system such modifications may comprise operations including translate, rotate, scale, extrude, fillet, chamfer, create feature, delete feature, change constraint, change visibility, mirror, pattern, loft, sweep, revolve, shell. Boolean, split, edit sketch, change material, or combinations thereof.

[0054] At step 330, following detection of the interaction, a message is communicated from the environment of the first interactive element to the environment of the second interactive element of the pairing. The message comprises a pairing identifier and state data associated with the first interactive element. The state data comprises updated parameter values for a set of parameters shared by the first interactive element and the second interactive element. In some examples, the messages may be JavaScript Object Notation (JSON) messages. In other cases, different message formats may be implemented. The pairing identifier and state data are appended to the message for transmission between environments.

[0055] At step 340, the message is received in the environment of the second interactive element. The receiving environment processes the incoming message and extracts the pairing identifier and state data for further processing. The communication occurs in real-time, enabling immediate synchronization between the paired interactive elements.

[0056] At step 350, the second interactive element is identified based on the pairing identifier. The receiving environment uses the pairing identifier to locate the corresponding interactive element from the list of paired interactive elements maintained in that environment.

[0057] At step 360, the state of the second interactive element is dynamically modified based on the state data. The receiving environment updates the parameters of the second interactive element using the updated parameter values contained in the state data. In the case of a geometric model in a modelling system, when the interactive element's parameters are updated, these changes propagate to the associated model components, causing visual and functional modifications to appear in the receiving environment. For example, when position parameters are updated the associated geometric entities move to new positions within the model display. This modification ensures that both environments maintain consistent representations of the same model data.

[0058] The method 300 supports bidirectional synchronization by enabling the same sequence of operations to occur from the environment of the second interactive element. When an interaction is detected with the second interactive element, the method 300 communicates a message from the environment of the second interactive element to the environment of the first interactive element, with the message comprising the pairing identifier and state data associated with the second interactive element. The message is received in the environment of the first interactive element, and the first interactive element is identified and dynamically modified based on the received state data.

[0059] Figure 4 is a diagram showing an example of a bidirectional messaging framework 400 between non-immersive and immersive environments. The messaging framework 400 may be implemented, for example, using classes in an object-oriented programming language. The messaging framework comprises two main architectural components that mirror each other across the non-immersive and immersive environments. Each environment implements a parallel structure of classes and objects that work together to maintain synchronized states between paired interactive elements.

[0060] In the non-immersive environment, a first class 405 represents an interactive element that enables user interaction in the non-immersive environment. The first class 405 may be instantiated in response to user input in the non-immersive environment. The first class 405 is associated with a first book-keeper class 410 that tracks paired objects for the non-immersive environment and facilitates communication with the immersive environment.

[0061] The first book-keeper class 410 includes a list of paired interactive elements 415 that stores references to all paired interactive elements within the non-immersive environment along with their associated pairing identifiers. The list of paired interactive elements 415 functions as a data structure that enables rapid lookup and identification ofinteractive elements based on their pairing identifiers. Each entry in the list of paired interactive elements 415 contains the pairing identifier and a reference to the corresponding interactive element, enabling the system to route messages to the correct recipient objects.

[0062] A communicator 420 is integrated within the first book-keeper class 410 to handle the transmission and reception of messages between the non-immersive and immersive environments. The communicator 420 implements the communication interface that enables the first book-keeper class 410 to send and receive messages containing state data and pairing identifiers. The communicator 420 manages transmission protocols, and message delivery and reception for the non-immersive environment.

[0063] In the immersive environment, a second class 425 represents the corresponding interactive element that mirrors the functionality of the first class 405. The second class 425 enables user interaction through controllers and maintains synchronized state with the first class 405. The second class 425 is associated with a second book-keeper class 430 that provides equivalent functionality to the first book-keeper class 410 within the immersive environment.

[0064] The second book-keeper class 430 comprises a list of paired interactive elements 435 and a communicator 440 that maintains references to all paired interactive elements within the immersive environment along with their associated pairing identifiers. The list 435 provides the same lookup and identification functionality as the list 415, enabling the immersive environment to route incoming messages to the appropriate interactive elements based on pairing identifiers.

[0065] A communicator 440 is integrated within the second book-keeper class 430 to handle message transmission and reception between the immersive and non-immersive environments. The communicator 440 implements the same communication interface and messaging format as the communicator 420, to enable bidirectional message exchange between the two environments.

[0066] The class diagram 400 illustrates the message flow between environments through three distinct message types. When the first class 405 is instantiated in the non-immersive environment, a first message 445 with state data is transmitted from the first class 405 to the first book-keeper class 410. The first book-keeper class assigns a pairing identifier to the interactive element represented by the first class 405 and stores the pairing identifier in the list 415. The pairing identifier is also appended to the message and the resulting message 455 is passed to the communicator 420 and sent to the communicator 440 of the second book-keeper class 430. The second book-keeper class 430 checks if there is any existing interactive element with the same pairing identifier in the list 435. If not, then an interactive element and its corresponding class will be instantiated as a second class 425. The second class 425 is populated based on the state data read from a received message 450.

[0067] When an interaction with the interactive element associated with the second class 425 is subsequently detected in the immersive environment, the immersive environment becomes the driving environment for change. In this case the message flow is reversed, with a message being transmitted from the second class 425 to the second book-keeper class 430. The message will contain the same unique pairing identifier that the second class 425 was previously assigned. The message, appended with the pairing identifier, is transmitted, via the communicator 440, to the communicator 420. The first book-keeper class 410 identifies the corresponding first class 405 based on the pairing identifier and the updated state data is propagated to the first class 405 to implement the interaction in the non-immersive environment.

[0068] Figure 4 illustrates a schematic representation of an exemplary bidirectional messaging framework according to one embodiment of the present disclosure. It will be appreciated by those skilled in the art that the specific architectural topology, constituent components, and data structure organizations depicted herein are illustrative only and nonlimiting. In alternative embodiments, various modifications, rearrangements, and substitutions of the architectural elements may be implemented without departing from the scope of the present disclosure.

[0069] The present disclosure is described with reference to flow charts and / or block diagrams of the method, devices and systems according to examples described herein.Although the flow diagram described shows a specific order of execution, the order of execution may differ from that which is depicted. In some examples, some blocks of the flow diagram may not be necessary and / or additional blocks may be added. It shall be understood that each flow and / or block in the flow charts and / or block diagrams, as well as combinations of the flows and / or diagrams in the flow charts and / or block diagrams can be realized by machine readable instructions.

[0070] The machine-readable instructions may, for example, be executed by a general-purpose computer, a special purpose computer, an embedded processor or processors of other programmable data processing devices to realize the functions described in the description and diagrams. In particular, a processor or processing apparatus may execute the machine-readable instructions. Thus, modules of apparatus may be implemented by a processor executing machine-readable instructions stored in a memory, or a processor operating in accordance with instructions embedded in logic circuitry. The term 'processor' is to be interpreted broadly to include a CPU, processing unit. ASIC, logic unit, or programmable gate set etc. The methods and modules may all be performed by a single processor or divided amongst several processors. Such machine-readable instructions may also be stored in a computer readable storage that can guide the computer or other programmable data processing devices to operate in a specific mode.

[0071] Figure 5 shows an example of a data processing system 500 comprising a processor 510 and a memory 520. The memory 520 comprises computer readable instructions 530 that are executable by the processor 510. In one case, the instructions 530, when executed by the processor 510, may cause the processor 510 to execute the steps of the method 300 performed from the non-immersive environment. In another case, the instructions 530, when executed by the processor 510, may cause the processor 510 to execute the steps of the method 300 performed from the immersive environment.

[0072] The present disclosure can be embodied in other specific apparatus and / or methods. The described embodiments are to be considered in all respects as illustrative and not restrictive. In particular, the scope of the disclosure is indicated by the appended claims rather than by the description and figures herein. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

CLAIMS1. A computer-implemented method of synchronizing an interaction in an immersive environment and a non-immersive environment of a software application, the method comprising:forming a pairing between an interactive element in the immersive environment and an interactive element in the non-immersive environment;detecting an interaction with a first interactive element of the paired interactive elements in either the immersive environment or non-immersive environment; andin real-time, in response to the detecting of the interaction:communicating a message from an environment of the first interactive element to an environment of a second interactive element of the paired interactive elements, wherein the message comprises a pairing identifier and state data associated with the first interactive element;receiving the message in the environment of the second interactive element;identifying the second interactive element based on the pairing identifier; anddynamically modifying a state of the second interactive element based on the state data.

2. The method of claim 1, wherein the software application is a modelling application.

3. The method of claim 2, further comprising:displaying a model in the non-immersive environment; anddisplaying a corresponding model in the immersive environment.

4. The method of claim 3, wherein the detecting of the interaction comprises detecting a change of state of the first interactive element resulting from a model modification initiated in the environment of the first interactive element.

5. The method of claim 4, wherein the state data comprises updated parameter values for a set of parameters shared by the first interactive element and the second interactive element, in response to the model modification.

6. The method of claim 5, wherein the dynamically modifying of the state of the second interactive element comprises updating the corresponding parameters of the second interactive element based on the updated parameter values, resulting in a corresponding modification to the model in the environment of the second interactive element.

7. The method of claim 4, wherein the model modification comprises one or more operations comprising: translate, rotate, scale, extrude, fillet, chamfer, create feature, delete feature, change constraint, change visibility, mirror, pattern, loft, sweep, revolve, shell, Boolean, split, edit sketch, change material, or a combination thereof.

8. The method of any of claims 1 to 7, further comprising:detecting an interaction with the second interactive element; andin real-time, in response to the detecting of the interaction with the second interactive element:communicating a message from the environment of the second interactive element to the environment of the first interactive element, wherein the message comprises the pairing identifier and state data associated with the second interactive element;receiving the message in the environment of the first interactive element;identifying the first interactive element based on the pairing identifier; anddynamically modifying a state of the first interactive element based on the state data.

9. The method of any one of claims 1 to 7, wherein the synchronization is bi-directional such that interactions initiated in either the immersive environment or the non-immersive environment are applied in the other environment in real-time.

10. The method of any one of claims 1 to 7, further comprising, for each interactive element of the paired interactive elements:storing the current state data of the respective interactive element and the pairing identifier in a data structure associated with the respective interactive element; and updating the data structure based on data received from the environment of the other interactive element.

11. The method of claim 1, wherein the pairing is maintained in each environment in a list of paired interactive elements that stores the pairing identifier and a reference to the corresponding interactive element.

12. The method of claim 1, wherein the message is a JavaScript Object Notation (JSON) message, andwherein the pairing identifier and the state data are appended to the JSON message.

13. The method of claim 1, further comprising:instantiating an interactive element in response to an initial user interaction in a first immersive environment of the immersive environment and the non-immersive environment.

14. The method of claim 13, further comprising:instantiating an interactive element in a second immersive environment of the immersive environment and the non-immersive environment in response to an initial message received from the first immersive environment.

15. A system for synchronizing an interaction in an immersive environment and a non-immersive environment of a software application, the system comprising:a first device to implement the non-immersive environment, the first device comprising:a data processor,memory; anda communication interface; anda second device to implement the immersive environment, the second device comprising:a data processor;a memory anda communication interface,wherein the memories of the first device and the second device are configured to store instructions that, when executed by the respective data processors of the first device and the second device, cause the system to perform the method according to any one of claims 1 to 14.

16. The system of claim 15, wherein the software application is a modelling application.

17. The system of claim 15 or 16, wherein the second device comprises:an immersive headset comprising a display configured to present the immersive environment to a user; anda user input device communicatively coupled with the immersive headset.

18. A device comprising:a processor configured to implement a non-immersive environment of a software application;a memory; anda communication interface,wherein the memory is configured to store instructions that, when executed by the processor, cause the device to execute the steps of any one of claims 1 to 14, performed from the non-immersive environment.

19. A device comprising:a data processor configured to implement an immersive environment of a software application;amemory; anda communication interface,wherein the memory stores instructions that, when executed by the processor, cause the device to execute the steps of any one of claims 1 to 14, performed from the immersive environment.

Images