Method for generating manufacturing references in immersive extended reality environments (immersive creator in the 8-sense system)

Abstract

[EN] A user is immersed in an immersive real-time environment (iXR, Immersive Extended Reality) with up to eight simultaneous sensory experiences. Specific elements of a target business (e.g., winery, gourmet restaurant) can be experienced as conventional images or processed using AI (engine 1, 2). During the immersive experience / flow, the user can optionally develop references / hints for new / improved products using gestures / acoustic / haptic instructions (immersive creator). A computer unit (RE ST, Ct.) records the user commands with cameras / microphones / imm. clothing, parameterizes the desired sensory impressions, and saves references for immediate or later commercial use.

Description

[0001] Terms, abbreviations, PAVE-S, examples 8 senses: XR Extended Reality (umbrella term, extended reality; includes VR, AR, MR) VR Virtual Reality (completely virtual representation of reality) AR Augmented Reality (augmented reality; virtual objects overlay the real world) MR Mixed Reality i immersive (immersing yourself in something, using computers, screens, projectors, cameras, audio, haptic / olfactory units). iXR-EE immersive Extended Reality real-time event EO Event Owner (person who, similar to a disc jockey, provides comprehensive sensory content in the iXR) Au Auditorium with at least 1 user in iXR-EE AV Audiovisual S Sensors 8Si-In 8 senses input (max. possible sensory stimulation) b2b business-to-business b2c business to customer RE ST computer and control unit Ct Counter / Counter of movements / Default parameters for target products / target processes BT Bluetooth 2 / 3D 2 / 3 dimensional PAVE-S Product-specific audio-visual ensemble (focus of the application), enhanced to include all remaining human senses. Achieving the effect with images, videos, acoustics, haptics, taste, and smell

[0002] Examples PAVE-S (wine / food): - Viticulture assets, such as vineyards / vines, wine cellars, halls, equipment, vehicles, kitchen. - Tools / aids / tools, such as pruning shears, bottles, glasses, cooking utensils, pots. - Products and ingredients, such as finished wine / sparkling wine, yeasts, clay minerals (bentonite), secondary ferment bacteria (biological acid degradation). - People involved in the entire process, pictures / videos (living and deceased) - Tracking data of people in the process or terroir under consideration - Visualizations of typical flavors and aromas of the final products (e.g. in viticulture: grapefruit, apple, lemon, cherries, vanilla, red wild berries, etc.) - Taste / smell / appearance (end product, e.g. glass filled with sparkling wine or wine) - Odors (control via atomizer unit) - Haptics (control via haptic clothing / haptic gloves or partially modulated air pressure) e.g. grape / grip simulation. - Crème brûlée; sugar / vanilla flavors; gas burner during caramelization.

[0003] The PAVE-S is stored on a file library of the computer unit used ( Fig. 1, RE ST) is stored (static / default, for example developed with companies; e.g. winery, gourmet restaurant) and can be flexibly accessed in real-time events. List of the 8 senses

[0004] Examples of associated measured values / parameters; the input characteristics in the iXR (machine input and / or human user) are explained in parentheses in the following section; Stimulus / examples of output and user change requests in the iXR that elicit responses. The sensory stimulations relate to the 8 known human senses (adapted from https: / / www.paedsinapod.com.au / sensory-processing-and-our-8-senses-explained-yes-8-not-5 / ).

[0005] Audio-Visual (AV): 1. Visual input (vision, visible light spectrum, projector): We see through our eyes. They capture light rays that create tiny images on the back of our eyeballs. Our brain interprets the signals it receives from the eyeball and tells us what we see. Stimulus in the iXR (changes the display image): Gestures user, camera). 2. Auditory input (auditory, spectrum of audible frequencies, loudspeaker): We receive acoustic input through our ears to assess whether it is important or just part of our everyday life, as well as where it comes from, how close it is, and whether we have heard it before. Stimulus: Speech, user, microphone.

[0006] Remaining sensors (S): 3. Taste input (taste, user; tasting product, atomizer): Our taste buds react to food and drinks. They tell us about taste, texture, and temperature. They are clustered in the mouth, tongue, and throat and receive five specific tastes: salty, sweet, bitter, sour, and umami (savory). Stimulus: User gestures. 4. Tactile input (touch; haptic clothing; air pressure modulation): Our tactile system helps us understand the important sensations of pressure, texture, heat and cold, and pain. This includes distinguishing between light and firm touch, as well as textures ranging from dry to wet and messy. Our tactile system is also linked to bonds and relationships. Stimulus: haptic clothing / gloves, partially modulated air pressure. 5. Olfactory input (smell, atomizer): The sensory receptors in our nose receive information about the smells around us. They transmit this information to the brain via a neural pathway. The power of smell can be underestimated. It is strongly linked to emotions and memory (neurobiologically) and can therefore trigger unexpected trauma reactions. Stimulus: Gestures / speech user. 6. Vestibular input (balance; user perception): These receptors are located in the inner ear and are stimulated by any change in head position, direction, or movement. Vestibular input contributes to our sense of body position in space, posture and muscle tone, maintaining a stable visual field, bilateral coordination, sense of balance, and awareness of gravity. Stimulus: User movement / gesture in the space in the immersive scene. 7. Proprioceptive input (movement, user sensation): This system is located in our muscles, tendons, ligaments, and joint receptors. It tells us where our body is in space and detects and controls force and pressure. It helps us feel grounded and know where we are and what we are doing. Stimulus: User movement in space in the immersive scene. 8. Interoceptive input (internal, user perception): The interoceptive system, sometimes referred to as the hidden sense, gives us the ability to sense what's happening in our bodies. It plays a role in influencing emotions and well-being and detects changes in our internal state. These include hunger and fullness, thirst, body temperature, heart and respiratory rate, social interactions, muscle tension, itching, nausea, drowsiness, and more. Stimulus: Holistic experience in the iXR room, measuring movement trajectories of the shoulders / arms and face. Patent description / details

[0007] Specifications for industrial processes and controls can be created using human gestures in virtual environments (reference patterns); methods are described as examples in WO002021216587A1, WO 2005 / 029306 A2, WO002018220438A3.

[0008] Public events featuring immersive real-time animations are also well-known, such as Claude Monet's Garden (https: / / www.monets-garten.de / , exhibition 2022, Düsseldorf / Mülheim) or the Frieda Kahlo exhibition (https: / / www.vivafridakahlo.de / , Hamburg 2023). Visitors can, for example, influence portraits immersively and immerse themselves in a virtual world using their own body movements and gestures.

[0009] Immersive hardware and software, such as XR hardware devices such as VR headsets, AR headsets, and MR headsets (see abbreviations), are becoming increasingly affordable and accessible. These technologies are being applied in areas such as gaming, education, training, and entertainment. In education, they are used to impart knowledge and skills. In entertainment, they are used to create new and innovative experiences. They offer opportunities for communication and collaboration.

[0010] What is new in this invention is the representation / provision of real and / or distorted images of products, product properties, or industrial manufacturing processes in an experience space, the immersive extended reality (iXR) space or immersive creator space. Inputs such as images, videos, pixel animations, smells, and haptic sensory impressions of the use of target products (wine, sparkling wine, food) are provided there. This allows a maximum of all eight human senses to be stimulated. In the immersive flow (movement in the immersive, holistic sensory flow), the user can creatively and intuitively make changes, for example, to pixel animations in real time, through movements / gestures or speech. These changes are specifications for product manufacturing or product properties. They are generated in real time and optionally documented.The advantage here is the creation of traceable references that can be used in a variety of commercial applications in research and development, production, marketing and sales, as well as in digital asset management. The potential for broad application by everyone, the completely digital implementation, and the accessibility on the Internet / Web 3 create significant leverage in dissemination and commercial use (. Fig. 1).

[0011] iXR room features and components: Projector and / or MX headsets, screen (SC), camera / webcam (CAM), microphone (Mic), computer / storage unit (RE ST, Ct.), lighting, music, participating event owner (EO), smartphone use (SMF) as a QR code reader or music streaming. The audience / user (Au) with at least one participant (possibly with haptic clothing) is an "immersive creator" (professional or layperson) with PAVE-S real-time interactions. Definition and use of the immersive extended reality space (iXR space)

[0012] The advantage of this invention is that familiar pleasure experiences (wine tasting with simple smell and taste) are enhanced with additional components, in the sense of holistic sensory perception, thus enabling a more intense experience for the user. Furthermore, the end customer is given the opportunity to be part of the production process or to influence it. This immersive "insider" experience enables a new approach to innovative product design. Extensions to commercial use for other industrial products of all kinds are generally possible.

[0013] In the iXR, the user (Au) / end customer and / or the EO can perceive their person / body parts, captured with at least one camera, together with PAVE-S elements (as a clearly visible image, as in conventional film recordings, or as a person / outline / glass avatar / pixel animation "converted" in a computer system (RE ST Ct), at least audiovisually and immersively. For example, hand gestures create flow-like movements in the displayed image (similar to wind moving snowflakes).

[0014] For example, the number and intensity of circular, stirring-like movements of the user (Au, Fig. 1, iXR) is a reference for real stirring movements, using a batonnage rod or an electric (barrique) stirrer in a barrel of young wine. Proper stirring is known to lead to improved quality (finer texture, better creaminess) in the final product, wine or especially sparkling wine. Input and sources

[0015] According to Fig. 2, the EO can apply 2 input methods as 8 sense input (8 Siln). 1. PAVE-S (engine 2, Fig. 2): classic images / videos / music, possibly as video animation. 2. New (engine 1, Fig. 2): Plug-in data processed by artificial intelligence, from suitable camera / sensor data of animated PAVE-S content (e.g. growth measurements of grapevines, grapes, terroir data of soils / environment / climate; Camera monitoring of fermentation / aging processes in wine barrels, measurement data from the intermediate / final product). Creation of references

[0016] In the immersive environment, the user can make inputs through gestures / body movements (flow situations), which can be evaluated by machine (RE ST Ct) and serve the specialist as desired specifications for manufacturing parameters.

[0017] The user moves in the iXR, his gestures etc. interact with the PAVE-S elements provided and are immediately visible in real time in the display medium.

[0018] Influence / wishes (e.g., users have the opportunity for immersive experiences worldwide and can set parameters immersively on-site (@home). Virtual batonnage then results in, for example, an individual texture and individual product creation. Decentralized iXR rooms (immersive for everyone)

[0019] Example: A winery ships a bottle of sparkling wine with a QR code. A customer configures their private iXR room at home using an app and existing resources (smartphone, PC with camera, Bluetooth speaker).

[0020] This allows for a continuous experience in video mode using a smartphone, PC, or projector. The app (according to Fig. 3), a recording mode (Trading NFT) is available. Economic use as a pure experience space as an immersive basic experience

[0021] As a basic version of the present process, the iXR serves as a pure real-time experience space, without the generation / referencing of manufacturing references. However, due to the holistic sensory perception and the associated emotional connection with the manufacturing process and / or the final product, it can nevertheless be economically beneficial. Furthermore, the user can be positioned to further develop products immersively within the immersive basic experience. Use of immersively generated references in the production

[0022] Parameters / requests for product properties are provided centrally or decentrally in the iXR - from key players / decision-makers (traditional producers, wine / food professionals) - from end customers.

[0023] The expert / winemaker / cellar master / chef interprets the wishes and implements them as far as feasible and sensible from his professional point of view.

[0024] The reference can be applied in real time or at later times.

[0025] Storage of immersively produced references, Fig. 3.

[0026] Movement patterns are recorded and stored in a computer and storage unit (RE ST Ct., Fig. 1) stored as a reference sample and then documented as an NFT (blockchain) and / or as a BAR / QR / RFID chip (e.g., a wine bottle). The reference samples are available at any time for production, in markets (e.g., Open Sea), @home (individual iXR room), or at trade fairs. Triggers for recording on / off the references are initiated by the EO, or decentrally, by the customer in the app.

[0027] The advantage of playback is the greatly improved customer loyalty through an expanded, immersive sensory perception of end products.

[0028] During the iXR event, immersive sequences and end products (wine / champagne bottles and / or video sequences / immersive artworks) can be purchased / ordered immediately. For example, ordering via QR code on a smartphone; ads in the iXR (see PHTM New York event, https: / / www.phntm.com / projects / Ivcidia). economy

[0029] Advantages include cost savings through the creation of traceable reference patterns (standard modules for manufacturers) and improved customer loyalty, both BTC- and BTB-related, both in the immersive base experience and in immersive product development, based on the use of all 8 human senses. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] WO 002021216587A1

[0007] WO 2005 / 029306 A2

[0007] WO 002018220438A3

[0007] Cited non-patent literature

[0000] https: / / www.paedsinapod.com.au / sensory-processing-and-our-8-senses-

[0004] https: / / www.monets-garten.de /

[0008] https: / / www.vivafridakahlo.de / ,

[0008] https: / / www.phntm.com / projects / Ivcidia).

[0028]

Claims

[1] Methods for product development in immersive product experience environments, - with provision of an immersive Extended Reality room (iXR, Fig. 1), equipped with at least one camera (2D, 3D), computer and storage unit, lighting system, sound system, microphone, event owner (EO) person (controller). With at least one output device (projector, SC, loudspeaker, VR-MR glasses (2D, 3D, scent / atomizer, taste, haptic clothing / gloves / air pressure modulation). Optionally with typical elements of the target process (wine, food, pruning shears, bottle...), both physical or virtual. characterized by - Immersive perception of sensory impressions, max. of all 8 human senses, by at least one user, whereby user inputs elicit reactions / changes in the offered PAVE-S graphics / animations (inputs) in real time. [2] Method according to claim 1, - where user inputs (gestures, haptics, acoustics) serve as coding rules for product attributes in an immersive experience. (E.g., changing the taste of sparkling wine by stirring a real or virtual baton in the air; see table for further examples.) Fig. 4). [3] Method according to claim 1, - where user inputs (gestures, haptics, acoustics) in the immersive space serve as training, marketing or enjoyment experiences (e.g. stirring with a batonnage stick in the air, without coding requirements) or as immersive basic experiences (e.g. event). [4] Method according to claim 1, - in which the offered PAVE-S sensory impressions are generated and processed from PASE-S mass data by an AI (engine 1). [5] Method according to claim 1, - in which PAVE-S sensory impressions are played live, in real time, from a manufacturing process (cam, sensor, etc.). [6] Method according to claims 1-5, - where user input is stored locally (RE ST Ct.) or decentrally, e.g., as blockchain data (NFT), or as codes / links such as QR, BAR, NFC, or RFID on the product. Codes are typically applied to the end product (wine / champagne bottle) and can be accessed as marketing or enjoyment experiences (e.g., mobile stream on home screen or at cinemas / theaters / museums / trade fairs). [7] Method according to claim 6, - where the expert (e.g., winemaker, cellar master, chef) uses coded user inputs of any PAVE-S elements, within the scope of their professional approval expertise, as desired specifications for production parameters. Implementation in real time (as an immersive short circuit) is possible, or implementation at a later time (immersive dead time). [8] Method according to claim 6, - where the iXR can be displayed decentrally, after decoding the stored user inputs using a suitable app. For example, a QR code on a champagne / wine bottle enables an immersive @home experience (immersive experience / happy space). [9] Method according to claim 6, - where remote-capable actuators (portioners, temperature controllers, agitators) are located on a container / container (barrel, tank) and execute customer requests (with mandatory approval from a specialist) during product production. Example: Stirring during battering with an electric stirring unit. [10] Method according to claim 1, - where the (EO) is located separately from the (Au), e.g. in the next room during the wine tasting or is connected to the network (world wide access).

Images