System for interactive prototyping of product, corresponding method of operation and computer program product

Abstract

A system for interactive prototyping of a product, a corresponding method of operation and a computer program product. The system includes a computer, a VR or AR headset wearable by a user, a master device, and a slave device. The computer runs a 2D design software program for designing a 2D screen interface intended to be incorporated into the product, runs a 3D graphics engine for generating a 3D model of the product, and incorporates the 2D screen interface into the 3D model of the product to generate a virtual representation of the product. The headset is coupled to the computer and displays a virtual representation of the product. The master device is coupled to the computer to exchange data between the 2D design software program, the 3D graphics engine, and the master device. The master device includes a first wireless communication interface. The slave device includes a second wireless communication interface, and also includes an actuator and / or a sensor. Actuation data is transmitted from the 2D design software program and / or 3D graphics engine to the actuator via the master device, and control data is transmitted from the sensor to the 2D design software program and / or 3D graphics engine via the master device.

Description

Technical Field

[0001] This specification relates to systems and methods for creating interactive prototypes of products.

[0002] In particular, this system and method can be used for prototyping the interior and / or exterior of vehicles such as cars, but in principle they can be used for prototyping any type of product such as home appliances, industrial machines, etc. Background Technology

[0003] In the field of industrial design, the purpose of prototyping is to generate a prototype (e.g., a physical object) of the desired product that closely mimics the appearance and / or function of the final product and reproduces the real user experience in the most readily available way.

[0004] Traditionally, physical prototypes of products (e.g., the interior or exterior of a car) are typically constructed. However, creating a physical prototype is a lengthy and expensive process. Therefore, prototyping with the aid of virtual reality (VR) and / or augmented reality (AR) devices has become increasingly common. As disclosed in the same applicant's documents EP 4222728 A1 and EP 4231191 A1, such VR / AR prototyping tools can be used in conjunction with configurable physical prototypes to provide extended reality (XR) design systems.

[0005] Furthermore, modern products (e.g., automobiles) typically feature Human-Machine Interfaces (HMIs) that include screens and / or touchscreens. Currently, various design programs are available for creating 2D interfaces (e.g., designing and testing the appearance and functionality of in-vehicle screens or touchscreens), but they lack the convenience of verifying how these interfaces function in real-world scenarios. On the other hand, programs exist that allow 3D prototyping in VR and AR environments, but designing 2D interfaces within the framework of 3D prototyping tools is often complex or simply impossible.

[0006] Therefore, there is a need in the art for improved systems and methods for plug-and-play interactive prototyping of products, which facilitate the integration of screen interfaces for 2D design, environments for 3D design, and physical (hardware) components. Summary of the Invention

[0007] The purpose of one or more embodiments of this specification is to provide such an improved prototyping system.

[0008] According to one or more embodiments, such an objective can be achieved by a prototyping system having the features set forth in the appended claims.

[0009] One or more embodiments may relate to corresponding prototyping methods.

[0010] One or more embodiments may relate to a corresponding computer program product that may be loaded into the memory of at least one processing unit (e.g., an MCU or CPU) and includes software code portions for performing steps of the method when the product is run on at least one processing unit. As used herein, references to such computer program products are to be understood as equivalent to references to a computer-readable medium containing instructions for controlling the processing unit to coordinate the implementation of the method according to one or more embodiments. The reference to “at least one” processing unit is intended to emphasize the possibility of implementing one or more embodiments in a modular and / or distributed manner.

[0011] The claims are an integral part of the technical teachings provided herein regarding the embodiments.

[0012] According to one aspect of this specification, a system for interactive prototyping of a product includes a computer configured to run 2D design software programs for designing a two-dimensional screen interface intended to be incorporated into the product, run a 3D graphics engine for generating a three-dimensional model of the product, and incorporate the two-dimensional screen interface into the three-dimensional model of the product to generate a virtual representation of the product. The system includes a virtual reality or augmented reality head-mounted device coupled to the computer and wearable by a user, the head-mounted device being configured to display the virtual representation of the product. The system includes a master device coupled to the computer via one or more serial communication interfaces for exchanging data between the 2D design software programs, the 3D graphics engine, and the master device via software plug-ins running on the computer. The master device also includes a first wireless communication interface. The system includes at least one slave device including a second wireless communication interface for exchanging data with the first wireless communication interface. The at least one slave device also includes at least one actuator device and / or at least one sensor device. Actuation data is transmitted via the master device from the 2D design software programs and / or the 3D graphics engine to the at least one actuator device. Control data is transmitted via the master device from the at least one sensor device to the 2D design software programs and / or the 3D graphics engine.

[0013] Therefore, one or more embodiments facilitate the seamless integration of virtual 2D HMI interfaces into the 3D virtual model of a product, and their merging with the physical entity to improve the design verification phase.

[0014] According to another aspect of this specification, a method of operating a system for interactive prototyping of a product includes:

[0015] -Use computer-run 2D design software programs to design two-dimensional screen interfaces intended for integration into products;

[0016] - Use a computer-driven 3D graphics engine to generate a 3D model of the product;

[0017] - Incorporate the two-dimensional screen interface into the product's three-dimensional model to create a virtual representation of the product;

[0018] - A virtual representation of the product displayed on a virtual reality or augmented reality headset coupled to a computer and wearable by the user;

[0019] - The computer is coupled to the host device via one or more serial communication interfaces to exchange data between the 2D design software program, the 3D graphics engine and the host device via software plug-ins running on the computer;

[0020] - Data is exchanged between the master device and at least one slave device via the first wireless communication interface of the master device and the second wireless communication interface of the slave device;

[0021] - At least one actuator device that transmits actuation data from a 2D design software program and / or a 3D graphics engine to a slave device via a master device; and

[0022] - Control data is transmitted from at least one sensor device of the slave device to the 2D design software program and / or 3D graphics engine via the master device. Attached Figure Description

[0023] The invention will now be described in detail with reference to the accompanying drawings, which are provided by way of non-limiting example only, in which:

[0024] - Figure 1 This is an example of a circuit diagram of the main device used in the prototype manufacturing system according to the present invention;

[0025] - Figure 2 This is an example of a circuit diagram of a subordinate device used in a prototype manufacturing system according to the present invention;

[0026] - Figure 3 A circuit diagram illustration of another dependent device used in the prototype fabrication system according to the invention; and

[0027] - Figure 4 This is a block diagram illustration of a prototype manufacturing system according to the present invention. Detailed Implementation

[0028] As expected, this specification discloses a system and method designed to seamlessly connect three traditionally separate design environments: a 2D design environment (e.g., a software environment) for designing user interfaces (e.g., screens and touchscreens); a 3D design environment (e.g., a software environment) for designing 3D models of the shape and / or appearance of a product, where the 2D user interface is intended to be incorporated (e.g., a vehicle interior, such as a dashboard area), which can be visualized using VR / AR tools; and a physical (hardware) prototype or object that reproduces at least some of the shape and / or physical device of the 3D model. By doing so, designers can effortlessly develop a comprehensive and interactive prototyping environment that combines physical objects and digital (virtual) representations in a so-called “extended reality” (XR) prototyping system for validating designs.

[0029] Essentially, the design and validation process includes the design of a 2D screen interface (e.g., including screen layout, transitions, animations, etc.) designed to be incorporated into the product using traditional 2D interface design programs (e.g., ProtoPie). The design and validation process also includes designing a 3D model of the product (e.g., including shapes) using traditional 3D design programs (e.g., Unreal Engine). In addition to the 2D screen interface, the product may also include additional physical devices that users can interact with (e.g., buttons, knobs, etc.) and / or physical devices that can affect the user experience (e.g., LED lights). By merging the 2D interface, 3D model, and physical devices (e.g., using virtual reality or augmented reality headsets where the outputs of the 2D and 3D design environments are merged), the behavior of the 2D interface can be simulated along with the behavior of other devices (e.g., ambient light). For example, during vehicle design validation, designers can interact with a virtual touchscreen, reproduced by a VR / AR headset in a virtual 3D model of the vehicle's interior, overlaid on the physical object, to change some settings of the interior lighting. Then, the interior lighting can be modified accordingly in the simulated 3D model to provide designers with realistic feedback.

[0030] In use case scenarios, the design system according to this specification can be used in the automotive industry. Designers can sit in the driver's seat inside a virtual car and validate their designs in an immersive 3D VR / AR / XR environment. As the designer navigates the digital representation, he or she can simultaneously test form and function, envisioning the interaction between vehicle hardware and user experience (UX). While seated in the 3D VR / AR / XR environment, designers can seamlessly interact with 2D HMI prototypes dedicated to infotainment systems (e.g., touchscreens on the center console). The digital display reflects the dashboard controls, touchscreens, and interactive elements that the driver and passengers will interact with. The 2D screen interface is incorporated into the virtual 3D environment, allowing for comprehensive design validation. Additionally, the testing phase relies on the use of real (physical) in-vehicle controls, fabricated as physical prototypes within a physical vehicle. These controls provide designers with tangible feedback, facilitating realistic testing without requiring full production integration at this stage. The overlay of physical prototype controls and the digital 3D environment (including digital 2D interfaces) provides a realistic simulation of the tactile and functional aspects of real in-vehicle controls. This approach not only accelerates the design iteration process, but also allows for checking whether the HMI is seamlessly integrated into the vehicle's form factor.

[0031] In another use case scenario, the system designed according to this specification can be used in the aerospace and aviation industries. Here, HMI designers can step into a 3D simulation environment that recreates the cockpit, interact with 2D avionics control panels, and test aviation-specific physical controls. Immersive 3D VR / AR / XR environments allow verification that HMI designs meet stringent aviation requirements while providing realistic interaction between virtual and physical components. This testing includes the use of physical prototypes, allowing for realistic evaluation without the need for full production integration at this stage.

[0032] In another use case scenario, the system designed according to this specification can be used for the design of home automation and IoT devices. Designers working on smart home devices can connect real IoT devices to the prototype. This allows for testing the integration of physical devices with digital interfaces, thereby ensuring a seamless user experience when users interact with both the application and the physical hardware.

[0033] In another use case scenario, the system designed according to this specification can be used for the design of medical devices and healthcare systems. HMI designers can navigate through immersive 3D VR / AR / XR representations of advanced medical facilities. In the virtual space, designers can seamlessly utilize 2D HMI prototypes to test the usability of various 2D interfaces of medical devices, reflecting the actual interfaces that healthcare professionals will interact with in real-world scenarios. Also included in this context are physical prototyping controls, allowing for nuanced evaluation of how the HMI design meets the tactile and operational expectations of healthcare practitioners, even when not ready for production. The immersive 3D VR / AR / XR environment allows verification that the HMI design meets the stringent usability standards expected in healthcare settings and integrates seamlessly with the physical tools used by healthcare professionals. The inclusion of physical controls improves the realism of the testing phase.

[0034] Therefore, the systems and methods described herein can be used in the design of any industrial product, from transportation vehicles to appliances, machines, healthcare devices, and so on. For ease of explanation, the following text will primarily refer to the field of automotive design, and particularly the design of automotive interiors.

[0035] The prototyping system according to the present invention relies on a hardware platform that acts as a bridge connecting a 3D environment created within the framework of an industry-standard graphics engine (or rendering engine, 3D engine, game engine) with a complex 2D HMI prototype (e.g., screen layout and functionality) created within the framework of an industry-standard 2D HMI prototyping tool. Therefore, designers can leverage the advantages of both software environments (3D and 2D) without encountering compatibility issues or constraints. The bridging function of the hardware platform is provided by a bidirectional communication channel established between the 3D graphics engine and the 2D prototyping tool. Additionally, the hardware platform allows for the connection of hybrid 3D / 2D environments with real-world hardware components (e.g., using augmented reality headsets to overlay representations of 3D / 2D digital environments onto physical hardware objects). Thus, designers can test the integration of physical controls and devices within the prototyping environment. Physical controls and devices include, but are not limited to, buttons, switches, knobs, joysticks, and other tangible interfaces. Physical controls can be prototyping controls (i.e., not intended for full production), but in any case, they provide a realistic evaluation of the user experience. By relying on the integration of physical prototyping controls within a digital 3D / 2D AR environment, designers can depend on an iterative design process without requiring full production integration at each stage. This feature is particularly advantageous because it provides a practical approach to improving designs based on real-world interactions, accelerates the design iteration process, and ensures seamless coordination between hardware and software components.

[0036] The prototype manufacturing system according to the present invention includes a main PCB or main device operating as a master device, and one or more auxiliary PCBs or auxiliary devices operating as slave devices.

[0037] Figure 1 This is a circuit diagram illustration of a possible architecture for the main device 10. The main device 10 includes a PCB 11 coupled to a first serial communication module 12a (e.g., a USB module), a second serial communication module 12b (e.g., a USB module), a level shifter circuit 13, a wireless communication module 14 (e.g., a Bluetooth module), and a status LED 15. The first serial communication module 12a is coupled to a first communication channel of the PCB 11 (e.g., transmit pin TX1 and receive pin RX1 of the PCB 11), and is also coupled to the power supply pins GND (i.e., ground) and 5V (i.e., providing a 5V power supply voltage) of the PCB 11 to be powered thereon. The second serial communication module 12b is coupled to a third communication channel of the PCB 11 (e.g., transmit pin TX3 and receive pin RX3 of the PCB 11), and is also coupled to the power supply pins GND and 5V of the PCB 11 to be powered thereon. The level shifter circuit 13 has a high-voltage (e.g., 5V) communication channel coupled to a second communication channel on PCB 11 (e.g., transmit pin TX2 and receive pin RX2 of PCB 11), and a low-voltage (e.g., 3.3V) communication channel coupled to a communication channel on Bluetooth module 14 (e.g., transmit pin TXD and receive pin RXD of Bluetooth module 14). The level shifter circuit 13 is also coupled to power supply pins GND, 5V, and 3V3 (the latter providing a 3.3V supply voltage) on PCB 11 to be powered thereon, and is configured to shift and transmit signals between Bluetooth module 14 and PCB 11 (i.e., between the second communication channel on PCB 11 operating at 5V and the communication channel on Bluetooth module 14 operating at 3.3V). Bluetooth module 14 is also coupled to power supply pins GND and 3V3 on PCB 11 to be powered thereon. Status LED 15 can be, for example, an RGB LED, having a cathode terminal coupled to the power supply pin GND of PCB 11, and three anode terminals (one for each LED) possibly coupled via (discrete) resistors to corresponding control terminals (e.g., numbered 1, 2, 3) of PCB 11. Status LED 15 can provide visual feedback on the current status of the system, thereby enhancing user interaction and troubleshooting capabilities.

[0038] Figure 2This is a circuit diagram illustration of a possible architecture for slave device 20. Slave device 20 includes a PCB 21 coupled to a level shifter circuit 23, a wireless communication module 24 (e.g., a Bluetooth module), and an LED strip 25. The level shifter circuit 23 has a high-voltage (e.g., 5V) communication channel coupled to the PCB 21 (e.g., transmit pin TX1 and receive pin RX1 of PCB 21) and a low-voltage (e.g., 3.3V) communication channel coupled to the Bluetooth module 24 (e.g., transmit pin TXD and receive pin RXD of Bluetooth module 24). The level shifter circuit 23 is also coupled to the power supply pins GND, 5V, and 3V3 of PCB 11 (which provide ground, 5V, and 3.3V power supply voltages, respectively) to be powered thereunder, and is configured to shift and transmit signals between Bluetooth module 24 and PCB 21 (i.e., between the communication channel of PCB 21 operating at 5V and the communication channel of Bluetooth module 24 operating at 3.3V). Bluetooth module 24 is also coupled to the power supply pins GND and 3V3 of PCB 21 to be powered thereunder. LED strip 25 has a ground terminal coupled to the power supply pin GND of PCB 11, and a control terminal possibly coupled to a corresponding control terminal (e.g., numbered 1) of PCB 21 via a (discrete) resistor. Furthermore, LED strip 25 has a power supply terminal coupled to the positive terminal of battery 26 or battery pack (e.g., a 5V battery, possibly including one or more AAA batteries or a 5V power supply pack), and the ground terminal of LED strip 25 is also coupled to the ground terminal of battery 26. By incorporating a power supply, the slave configuration ensures portability and autonomy, allowing the LED strip to operate independently of the availability of an external power source.

[0039] Figure 3 This is a circuit diagram illustration of another possible architecture for slave device 20. Slave device 20 includes a PCB 21 coupled to level shifter circuit 23, wireless communication module 24 (e.g., Bluetooth module), first button 27, second button 28, and rotary encoder 29. Level shifter circuit 23 and Bluetooth module 24 are coupled to PCB 21, and between them, as previously referenced. Figure 2As described above. Button 27 has a ground terminal coupled to the power supply pin GND of PCB 21, and a positive terminal possibly coupled to a corresponding sensing terminal (e.g., numbered 1) of PCB 21 via a (discrete) resistor. Similarly, button 28 has a ground terminal coupled to the power supply pin GND of PCB 21, and a positive terminal possibly coupled to a corresponding sensing terminal (e.g., numbered 2) of PCB 21 via a (discrete) resistor. Rotary encoder 29 has a ground terminal coupled to the power supply pin GND of PCB 21, a positive terminal coupled to the power supply pin 5V of PCB 21, and detection terminals coupled to three corresponding sensing terminals (e.g., numbered 3, 4, and 5) of PCB 21.

[0040] From a functional and operational perspective, the prototype manufacturing system according to the present invention can be referenced. Figure 4 To describe it using a block diagram. Figure 4 This illustrates the data connections between various components of the system. The prototyping system 1 includes a master device 10 and one or more slave devices 20, which are coupled to corresponding hardware devices 25, 27, 28, 29 (e.g., such as...). Figure 2 and 3(As illustrated in the illustration). The master device 10 and slave device 20 exchange data between them using their wireless communication modules 14, 24 (e.g., Bluetooth modules). The master device 10 uses its first serial communication module 12a (e.g., USB module) to exchange data with the computer 40 and opens a first communication port (COM port) to communicate with the graphics engine GE. The master device 10 uses its second serial communication module 12b (e.g., USB module) to exchange data with the computer 40 and opens a second communication port (COM port) to communicate with the prototyping tool PT. During operation, the computer 40 runs both the graphics engine GE (e.g., an industry-standard game engine) and the 2D HMI prototyping tool PT (e.g., industry-standard 2D prototyping software). The graphics engine GE communicates with the master device 10 via a first software plug-in PL1, which acts as a bridge between the master device 10 and the graphics engine GE. Plug-in PL1 uses libraries or protocols (e.g., socket.IO) to incorporate communication and open a communication port (COM port), which serves as an interface to the communication module 12a of the host device 10, facilitating data exchange between the two ports using a single PCB. The first software plug-in PL1 also enables the output of the 2D prototyping tool to be streamed to the graphics engine GE as dynamic images, which can be applied to 3D models. Thus, the real-time output from the 2D prototyping tool PT is also visible in the VR / AR headset HS coupled to the computer 40. For example, the UI designed in the 2D prototyping tool PT is visible on a 3D virtual screen in the graphics engine GE. The 2D HMI prototyping tool PT communicates with the host device 10 via a second software plug-in PL2, which acts as a bridge between the host device 10 and the 2D HMI prototyping tool PT. Plug-in PL2 uses libraries or protocols (e.g., socket.IO) to incorporate communication (i.e., sending and receiving messages between the prototyping tool PT and the main device 10) and opens a COM port, which serves as an interface to the second communication module 12b of the main device 10. Therefore, the main device 10 is readily available when connected to the computer 40 via a serial communication port (e.g., USB), providing a seamless and convenient plug-and-play user experience. Furthermore, the computer 40 connects to a VR / AR head-mounted device HS (or head-mounted display), which receives graphical information from the graphics engine GE and / or from the 2D HMI prototyping tool PT to reproduce, on the display of the head-mounted device HS, a merging of the appearance of a 3D model of the vehicle interior and the 2D HMI screen within the vehicle interior.

[0041] In short, when the communication port between the master device 10 and the computer 40 is open, messages are transmitted between the two design tools (i.e., the 3D engine and the 2D tool), enabling them to communicate with each other. The implementation of wireless (e.g., Bluetooth) modules in the master and slave devices allows each slave device to send and receive messages from the master device, and the master device then sends and receives messages to and from the game engine GE and the 2D prototyping tool PT. This facilitates efficient data exchange and synchronization between software and hardware components. The number of slave devices 20 in system 1 can vary depending on the application or project requirements, and therefore the number of Bluetooth modules 14 coupled to the master device 10 can also be varied. For example, if multiple slave devices are in use, a corresponding number of Bluetooth modules will be coupled to the master device to ensure communication efficiency. Slave devices may not communicate directly with each other, but if necessary, they can exchange messages through the master device, which acts as an intermediary.

[0042] Additionally, depending on the application or project requirements, each slave device 20 is connected to the corresponding hardware. For example, one slave device can manage LED light strips (such as...). Figure 2 (as illustrated in the example), while another slave device can operate several buttons (such as...). Figure 3 (As illustrated in the example). The LED light strip can be controlled from the master device, thus providing various functions such as power management (on / off), color selection, and animation control. Therefore, the LED light strip can be controlled not only via signals from hardware components (such as buttons), but also via data from a 2D HMI prototype incorporated into the VR / AR / XR environment and operated by a user wearing a head-mounted device HS, or via data from the user's interaction with the 3D environment. The provision of both sensors (e.g., buttons, knobs) and actuators (e.g., the LED light strip) coupled to the slave device 20 thus enables interactive control of various parameters of the virtual model inside the car. For example, during the validation phase, the brightness of the LED light strip can be adjusted by the user acting on a physical knob or by acting on a virtual slider reproduced on a virtual screen by the head-mounted device HS. Similarly, the effect of this change can be reflected in the real environment (by actually adjusting the brightness of the LED light strip) and in the AR / VR / XR environment, thus providing real-time feedback within the 2D prototype.

[0043] Therefore, the inclusion of physical controls expands the range of control possibilities, allowing for precise adjustments and intuitive interaction with both physical and digital components. The master device 10, which provides communication with slave device 20 and computer 40, allows for user input from a variety of sources, including virtual reality environments, 2D prototypes, and direct hardware interaction. This diverse input ecosystem enables users to interact with the prototyping system 1 in multiple ways, providing flexibility and versatility during the prototyping process.

[0044] Another example of the functionality of the prototyping system described in this paper is support for voice control. For example, one of the slave devices 20 can be coupled to a microphone for capturing the user's voice. Voice input can be activated by the user by pressing a button (whether physical or virtual). Once the voice input function is activated, System 1 processes voice commands within the target environment, whether it is a virtual reality simulation or a 2D prototype.

[0045] Obviously, the construction details and embodiments can vary widely with respect to what has been described and illustrated by way of example, without departing from the scope of protection of the invention as defined in the appended claims.

Claims

1. A system (1) for interactive prototyping of a product, comprising: - A computer (40) is configured to run a 2D design software program (PT) for designing a two-dimensional screen interface intended to be incorporated into a product, run a 3D graphics engine (GE) for generating a three-dimensional model of the product, and incorporate the two-dimensional screen interface into the three-dimensional model of the product to generate a virtual representation of the product. - A virtual reality or augmented reality head-mounted device (HS) coupled to the computer (40) and wearable by a user, the head-mounted device (HS) being configured to display the virtual representation of the product; - A host device (10) is coupled to the computer (40) via one or more serial communication interfaces (12a, 12b) to exchange data between the 2D design software program (PT), the 3D graphics engine (GE) and the host device (10) via software plug-ins (PL1, PL2) running by the computer (40), the host device (10) also including a first wireless communication interface (14); - At least one slave device (20) includes a second wireless communication interface (24) for exchanging data with the first wireless communication interface (14), and the at least one slave device (20) further includes at least one actuator device (25) and / or at least one sensor device (27, 28, 29); The actuation data is transmitted from the 2D design software program (PT) and / or the 3D graphics engine (GE) to the at least one actuator device (25) via the master device (10); And control data is transmitted from the at least one sensor device (27, 28, 29) to the 2D design software program (PT) and / or the 3D graphics engine (GE) via the master device (10).

2. The prototyping system (1) according to claim 1, wherein the one or more serial communication interfaces (12a, 12b) include one or more USB modules.

3. The prototyping system (1) according to claim 1 or claim 2, wherein the one or more serial communication interfaces (12a, 12b) comprise: The first serial communication interface (12a) is configured to open a first communication port with the computer (40) to exchange data with the 3D graphics engine (GE); and The second serial communication interface (12b) is configured to open a second communication port with the computer (40) to exchange data with the 2D design software program (PT).

4. The prototyping system (1) according to any one of the preceding claims, wherein, The first wireless communication interface (14) and the second wireless communication interface (24) include a Bluetooth module.

5. The prototyping system (1) according to any one of the preceding claims, wherein, The at least one actuator device (25) includes at least one of a lighting module, preferably an LED light strip (25), and a speaker.

6. The prototyping system (1) according to any one of the preceding claims, wherein, The at least one sensor device (27, 28, 29) includes at least one of a button, switch, knob, lever, and microphone.

7. The prototyping system (1) according to any one of the preceding claims, wherein, The at least one slave device (20) includes a local power supply (26) configured to supply power to the at least one actuator device (25) and / or at least one sensor device (27, 28, 29).

8. The prototyping system (1) according to any one of the preceding claims, wherein, The main device (10) includes a status LED (15), preferably an RGB status LED, which is configured to provide visual feedback on the current status of the system (1).

9. A method of operating the system (1) according to any one of the preceding claims, the method comprising: -Use the 2D design software program (PT) running by the computer (40) to design a two-dimensional screen interface intended to be incorporated into the product; - A three-dimensional model of the product is generated using the 3D graphics engine (GE) running on the computer (40); - Incorporate the two-dimensional screen interface into the three-dimensional model of the product to generate a virtual representation of the product; - The virtual representation of the product is displayed on the virtual reality or augmented reality head-mounted device (HS) coupled to the computer (40) and wearable by the user; - The computer (40) is coupled to the host device (10) via one or more serial communication interfaces (12a, 12b) to exchange data between the 2D design software program (PT), the 3D graphics engine (GE) and the host device (10) via software plug-ins (PL1, PL2) running by the computer (40); - Data is exchanged between the master device (10) and the at least one slave device (20) via the first wireless communication interface (14) and the second wireless communication interface (24); - Transmitting actuation data from the 2D design software program (PT) and / or the 3D graphics engine (GE) to the at least one actuator device (25) via the main device (10); and Control data is transmitted from the at least one sensor device (27, 28, 29) to the 2D design software program (PT) and / or the 3D graphics engine (GE) via the master device (10).

10. A computer program product that can be loaded into the memory of at least one processing unit, and includes a software code portion that, when executed by the processing unit, causes the processing unit to perform the steps of the method of claim 9.

Images