Interactive Hoodware System and Method

Abstract

The foodware system includes an identifier coupled to an instrument configured to be used by a user in connection with food. The foodware system also includes a detector configured to generate data indicative of the position of the instrument, the movement of the instrument, or both, based on detection of the identifier. Further, the foodware system includes a controller communicatively coupled to the detector and configured to command a speaker to generate audio data based on the position of the instrument, the movement of the instrument, or both.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 341,308, filed May 12, 2022, entitled "INTERACTIVE FOODWARE SYSTEMS AND METHODS", which is hereby incorporated by reference in its entirety for all purposes.

Background Art

[0002] This section is intended to introduce the reader to various aspects of technologies that may be related to various aspects of the present disclosure. The discussion here is considered to be useful in providing the reader with background information that facilitates a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these descriptions are not an admission of prior art and should be read in light of this perspective.

[0003] Amusement parks generally include attractions that provide various experiences to users. For example, an amusement park can include various attractions such as roller coasters, drop towers, log flumes, etc. Some attractions include environments that provide effects such as auditory stimulation, tactile stimulation, visual stimulation, and / or other special effects, which helps to provide an immersive experience to users. An amusement park can include other types of attractions and / or facilities such as restaurants that provide a dining experience to users.

Summary of the Invention

[0004] Specific embodiments within the scope of the present invention as initially claimed are summarized below. These embodiments are not intended to limit the scope of the present disclosure; rather, these embodiments are only intended to provide a concise summary of the specific embodiments disclosed. In fact, the present disclosure can encompass various forms that can be similar to or different from the embodiments described below.

[0005] A foodware system according to one embodiment includes an identifier coupled to an instrument configured to be used by a user to interact with food. The foodware system also includes a detector configured to generate data indicating the position of the instrument, the movement of the instrument, or both, based on the detection of the identifier. The foodware system further includes a controller communicatively coupled to the detector and configured to command a speaker to generate audio data based on the position of the instrument, the movement of the instrument, or both.

[0006] A method for providing an immersive dining experience according to one embodiment includes identifying an interaction between food and a user via one or more processors and based on data obtained by one or more sensors. The method also includes projecting an image onto the food via a projector in response to the interaction between the food and the user.

[0007] A foodware system according to one embodiment includes one or more sensors configured to obtain data indicating an interaction between food and a user. The foodware system further includes a controller communicatively coupled to the one or more sensors and configured to command a projector to project an image onto the food in response to the interaction between the food and the user.

[0008] These and other features, aspects, and advantages of the present disclosure will be further understood upon reading the following detailed description with reference to the accompanying drawings in which like reference numerals throughout the drawings indicate like elements.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Mode for Carrying Out the Invention

[0010] One or more specific embodiments of the present disclosure will be described below. For the sake of brevity in the description of these embodiments, not all features of the actual implementation modes will be described in this specification. It should be understood that, as with any technical or design project, in the development of such an actual implementation configuration, numerous implementation-specific decisions need to be made to achieve the specific goals of the developer, which may vary for each implementation configuration, such as compliance with system- and business-related constraints. Furthermore, although such development efforts can be complex and time-consuming, it should be understood that for those skilled in the art having the advantages of the present disclosure, they are routine operations in design, fabrication, and manufacturing.

[0011] When introducing elements of various embodiments of the present invention, the articles "a", "an", and "the" are intended to mean that one or more of the elements are present. The terms "comprising", "including", and "having" are inclusive and mean that additional elements other than the recited elements can exist. In addition, it should be understood that references to "an embodiment" or "one embodiment" of the present invention are not intended to be construed as precluding the existence of additional embodiments that incorporate the same described features.

[0012] The present disclosure generally relates to the field of special effects for use in interactive environments such as amusement parks. More specifically, the present disclosure relates to systems and methods for generating auditory stimuli (e.g., binaural audio), tactile stimuli, and / or visual stimuli during a dining experience. As used herein, binaural audio can create a three-dimensional (3D) stereo sound sensation for a user based on how the human auditory system receives and processes sound. For example, if a dog barks in the user's left ear, the sound of the bark may take several seconds longer to reach the user's right ear compared to the left ear. Also, the sound of the bark may sound louder in one ear than the other. Binaural audio takes into account such differences in the time and intensity associated with the sound. By localizing sound based on the orientation and / or movement of the human head, binaural audio can provide an immersive experience for the user. For example, the user can hear a whisper in one ear or a bird flying overhead.

[0013] Furthermore, the tactile stimulus can cause a tactile sensation to the user, such as by applying force, vibration, and / or movement to the user. As used herein, the tactile stimulus can include aroma (e.g., olfaction). Furthermore, the visual stimulus can include a projected image (e.g., a two-dimensional [2D] image or a 3D image) that can be observed by the user (e.g., regardless of the presence or absence of viewing glasses). In one embodiment, projection mapping technology can be utilized to generate a projected image and project the projected image onto tableware (e.g., plates, tabletops) and / or food during a dining experience (e.g., an overlay image that creates virtual enhancement and / or virtual features on the tableware and / or food). Additionally or alternatively, projection mapping technology can be utilized to project a projected image onto any other type of display surface, such as a conventional movie screen. A 2D image refers to what is typically regarded as a "flat" image that appears to exist in two dimensions when viewed by the user, and a 3D image is also a "flat" image provided in a way that appears to exist in three dimensions when viewed by the user. Both 2D images and 3D images can be provided on non-planar surfaces, such as food, a textured tablecloth, etc.

[0014] With the above in mind, an amusement park or other facility can include a foodware system (e.g., an interactive foodware system) configured to create special effects by generating auditory, visual, and / or tactile stimuli during a dining experience. In fact, a specific hardware configuration (e.g., circuitry), software configuration (e.g., algorithm structure and / or modeled response), and combination of specific attraction features can be utilized to provide an immersive dining experience to the user.

[0015] The foodware system can include tableware having sensing components (e.g., sensors). Additionally or alternatively, the sensing components can be distributed with respect to the dining environment (e.g., in a location separate from the tableware). Additionally, the foodware system can include speakers for generating auditory stimuli (e.g., binaural audio), haptic devices for generating tactile stimuli, and / or projectors or other display devices for generating visual stimuli based on data obtained by the sensing components. The tableware can include plates, bowls, frying pans, containers, cups, utensils / cutlery, tabletops, napkins, etc. The foodware system can track various parameters of the tableware, food, and / or the user. For example, the foodware system can track food (e.g., edible food including solid food and / or liquid food such as water and other beverages), such as the movement of the food (e.g., the movement of the food with respect to the tableware, where the movement of the food is caused or imparted by the user, such as via cutlery). The foodware system can track the movement of the tableware (e.g., the movement of the cutlery with respect to the food, where the movement of the cutlery is caused or imparted by the user). The foodware system can then provide auditory, visual, and / or tactile stimuli based on the movement of the food and / or the movement of the tableware. The foodware system can determine the eating habits and patterns of the user (e.g., the types of food provided to and / or consumed by the user, the amount of each type of food provided to and / or consumed by the user, the amount of each type of food provided to the user but not consumed by the user, the consumption rate of each type of food by the user, the order of consumption when different types of food are presented to the user, and / or the sounds made by the user when eating each type of food). In such cases, the foodware system can provide auditory, visual, and / or tactile stimuli based on the eating habits and patterns of the user.

[0016] As a specific example to facilitate discussion, in response to determining that a user is operating a knife to slice a lump of bread on a plate, the headwear system can generate binaural audio that produces for the user the effect of hissing noise that appears to originate from the lump of bread and a visual image depicting a snake that appears to the user to move from the lump of bread onto the plate. It should be understood that the headwear system also temporally coordinates auditory, visual, and / or tactile stimuli with each other and with the detected movement (e.g., the movement of cutlery and / or food such as the movement of the knife and bread) to provide the user with an overall immersive effect.

[0017] To generate binaural audio and other effects in this way, the controller of the headwear system receives data from various sensing components (e.g., detectors and other sensors). Non-limiting examples of detectors can include radio frequency identification (RFID) readers, near field communication (NFC) readers, barcode scanners, quick response (QR) code scanners, and / or cameras. The detectors can be communicatively coupled to the controller of the headwear system. Similarly, non-limiting examples of identifiers detectable by the detectors can include RFID tags, NFC tags, barcodes, and QR codes. Some additional non-limiting examples of identifiers detectable by the detectors can include visual patterns, retroreflective materials having specific colors (e.g., visible or invisible to humans), and the shapes of materials or items (e.g., unique edges along tableware). In one embodiment, each identifier can be coupled and / or integrated with each piece of tableware. For example, one identifier such as an RFID tag can be coupled to the knife used to slice a loaf of bread. A detector such as an RFID reader can identify the position and / or movement of the knife based on the detection of the identifier (e.g., communication with the identifier). The detector can then transmit position data and / or movement data related to the knife to the controller. The controller can process the position data and / or movement data related to the knife to determine that the movement of the knife corresponds to a cutting or slicing operation (e.g., by referring to a stored library of movements and / or classifying the movement of the knife via one or more algorithms). Next, as described herein, the controller can create an immersive dining experience for the user by instructing the speakers to generate binaural audio and instructing the projector to project visual images, etc.

[0018] The controller can process data generated by and received from detectors and / or other sensors to track users (e.g., user actions), food (e.g., food type, food location, food displacement, food weight), and / or tableware (e.g., cutlery, cutlery movement). For example, the controller can utilize any of various tracking technologies and tools such as RFID monitoring, NFC monitoring, proximity sensors, coordinate grids, cameras, motion sensors, food displacement sensors, food weight sensors, color markers, computer vision, etc. Based on data related to users, food, tableware, and the dining environment, the generator can effectively generate responsive and / or interactive auditory stimuli, tactile stimuli, and / or visual stimuli during the dining experience. Further, the controller can transmit the data and / or related information to computer devices, servers, remote servers, etc. related to the staff of the restaurant or dining attraction. The data and / or related information can be useful when the restaurant prepares and customizes dishes according to the eating habits and patterns of the users. For example, in addition to tracking the types of food ordered, the restaurant can also gain insights from the types of food ordered but not consumed by the user (e.g., food left on the plate).

[0019] Referring to the figures, FIG. 1 shows a block diagram of an embodiment of a headwear system 10 (e.g., an interactive headwear system). As shown, the headwear system 10 can include a detector 12, a controller 14 (e.g., a programmable logic controller or a computer), tableware 20, a speaker 24, a display device 26, and / or a haptic device 28. It should be understood that the headwear system 10 may omit certain elements shown in FIG. 1 and / or may include additional elements such as lighting elements (e.g., light emitting elements) that provide special lighting effects. The controller 14 can include a processor 16 and a memory device 18. Further, each item of the tableware 20 can be coupled to a respective identifier 22.

[0020] The detector 12, the tableware 20 (e.g., including respective identifiers 22), the speaker 24, the display device 26, and / or the haptic device 28 can be communicatively coupled to the controller 14. The detector 12 can include any suitable type of detector configured to detect the respective identifiers 22. More specifically, the detector 12 can include any suitable type of detector configured to generate data (e.g., a signal) indicative of the position and / or movement of the tableware 20 via detection of the respective identifiers 22. For example, the detector 12 can include a radio frequency identification (RFID) reader, a near field communication (NFC) reader, a barcode scanner, a quick response (QR) code scanner, a camera, or any combination thereof. In such cases, the identifier 22 can include a corresponding detectable or readable identifier such as an RFID tag, an NFC tag, a barcode, and a QR code, or any combination thereof. Additionally or alternatively, the identifier 22 can include a visual pattern, a retroreflective material having a specific color (e.g., visible or invisible to humans), and / or the shape of a material or article (e.g., a unique edge along the tableware).

[0021] Accordingly, in operation, one identifier 22 can be coupled and / or integrated with a first element (e.g., a fork) of the tableware 20, and / or another identifier 22 can be coupled and / or integrated with a second element (e.g., a plate) of the tableware 20, and so on. The detector 12 can then generate data indicating the respective positions and / or respective movements of the first element of the tableware 20 and / or the second element of the tableware 20 via detection of the identifier 22. In particular, the detector 12 can generate data indicating the positions and / or movements of the first element of the tableware 20 and the second element of the tableware 20 by periodically and / or continuously detecting and tracking the identifier 22 (e.g., in a three-dimensional [3D] space within an environment such as a show environment or a dining environment).

[0022] Detector 12 can provide data indicating the position and / or movement of the tableware 20 to the controller 14 periodically and / or continuously during an experience (e.g., a show experience or a dining experience). The controller 14 can execute instructions to process the data to determine the position and / or movement of the tableware 20. In one embodiment, the controller 14 can refer to one or more databases (e.g., lookup tables) to identify a particular piece of tableware 20 associated with the identifier 22. Further, the controller 14 can also refer to one or more databases to identify at least currently the user associated with the particular piece of tableware 20 and / or the identifier 22. For example, a user can associate the identifier 22 (e.g., a band) with the identifier 22 when purchasing it, and then the user may reuse the identifier 22 on multiple different pieces of tableware 20 over time (e.g., attach it to a fork during one experience and then attach it to another fork during another experience). As another example, a user can associate the tableware 20 with the tableware 20 and the identifier 22 of the tableware 20 at the time of purchase of the tableware 20 (e.g., integrated into an element of the tableware 20 or otherwise fixed to an element of the tableware 20), and then the user can reuse the tableware 20 over a long period of time. As yet another example, the identifier 22 can be incorporated into an element of the tableware 20 or reused with an element of the tableware 20, and the elements of the tableware 20 are reused by multiple different users over time. In such cases, the elements of the tableware 20 and the identifier 22 of the elements of the tableware 20 can be temporarily assigned to or associated with the user during an experience (e.g., when sitting at a table).

[0023] In this way, the controller 14 can access information regarding the characteristics of specific elements of the tableware 20 (e.g., the type such as a knife or a plate, etc.) and / or historical data related to the user's previous experiences (e.g., the user has positively reacted to a specific special effect). However, it should be understood that the technology disclosed herein can be implemented without associating the tableware 20, the identifier 22, and / or the user with each other. For example, the detector 12 can detect any identifier 22 within the range and / or utilize other sensors, and then the controller 14 can control other elements accordingly.

[0024] According to one embodiment, the controller 14 can execute instructions to control the speaker 24 to generate audio data (e.g., binaural audio), control the display device 26 to generate visual images, and / or control the haptic device 28 to generate a haptic output based on the identifier 22 received from the detector 12, the position of the tableware 20, and / or data indicating the movement of the tableware 20. Further, the controller 14 can receive and process other types of data to determine the timing and manner of operating the speaker 24, the display device 26, and / or the haptic device 28. For example, the controller 14 can receive and process data indicating the type of the tableware 20, the type of food provided to the user, the position of the food (e.g., the food position of the food), the movement of the food (e.g., the food movement of the food), the position of the user (e.g., the user position of the user), the movement of the user (e.g., the user movement of the user), and / or the body posture of the user (e.g., the orientation of the user's head). Also, the controller 14 can derive other information from the data, such as the relative position of the food and the tableware 20 (e.g., the relative movement of the tableware 20 with respect to a specific food), and / or the interaction between the food and the tableware 20 (e.g., the contact between a specific food and the tableware 20).

[0025] As shown, the foodware system 10 can include a detector 12 configured to detect an identifier 22 (e.g., via radio frequency communicator, code reading technology, and / or image processing technology), and one or more other sensors 30 operative to collect other types of data. The one or more sensors 30 can include a camera, a motion sensor, a food weight sensor, a food displacement sensor, or any combination thereof. It should be understood that the one or more sensors 30 can collect the data described herein via any suitable technique. For example, the detector 12 may be configured to detect the identifier 22 (e.g., obtain a unique identifier of an RFID tag or other code), thereby enabling association with specific foodware 20 and / or a user. Also, the detector 12 can continue to detect the identifier 22 over time to effectively track the position and / or movement of the identifier 22 and the foodware 20. Additionally or alternatively, a camera (which may be considered the detector 12 for detecting the identifier 22 and / or operate in addition to the detector 12 that tracks the tag or other code identifier 22 via techniques other than image processing techniques) can track the position and / or movement of the foodware 20. For example, the camera can capture an image of the environment and provide the image of the environment to the controller 14. The controller 14 can then apply computer vision techniques (e.g., template matching, item recognition, gesture recognition) to track the position and / or movement of the foodware 20. In fact, image analysis can be used to determine any of a variety of data including the type of foodware 20, the type of food provided to the user, the position of the food, the movement of the food, the position of the user, the movement of the user, and / or the posture of the user. Further, it should be understood that the detector 12 can be omitted and / or any of the one or more sensors 30 can be utilized alone or in any combination to collect any of the data described herein.

[0026] Also, it should be understood that one or more sensors 30 can include one or more proximity sensors (e.g., capacitance sensors) disposed on the tableware 20. The one or more proximity sensors can be configured to detect an object, such as via a detectable tag (e.g., a metal tag, a magnet) embedded in the object. As an example, the one or more proximity sensors can be disposed on a tabletop and configured to detect a food container (e.g., a plate or a cup) when the food container is within the range of the one or more proximity sensors. As another example, the one or more proximity sensors can be disposed on a plate and configured to detect cutlery (e.g., utensils) when the cutlery is within the range of the one or more proximity sensors. In such cases, the data obtained by the one or more proximity sensors indicates the position and / or movement of one article relative to another article (e.g., cutlery relative to a plate). The one or more proximity sensors can transmit the data to the controller 14 to facilitate the processes and techniques disclosed.

[0027] It should be understood that a plurality of proximity sensors can be arranged around the plate, such as a first proximity sensor in the first quadrant of the plate and a second proximity sensor in the second quadrant of the plate. Further, it can be known (for example, input to the controller 14 via an operator and / or via analysis of an image captured by a camera) that the first quadrant contains a first food and the second quadrant contains a second food. Next, when cutlery is detected by the first proximity sensor, the controller 14 can initiate a first special effect, such as the sound of laughter and an image of a hyena. Next, when cutlery is detected by the second proximity sensor, the controller 14 can initiate a second special effect, such as an image of a dragon and a vibrating tactile stimulus. In this way, the special effects can respond to the position and / or movement of cutlery-type tableware, the interaction between different elements of the tableware 20, the interaction between the tableware 20 and the food, and the type of food. It should be understood that one or more sensors 30 can be arranged in other ways, such as on cutlery-type tableware. For example, a proximity sensor that detects features of the plate, a camera that captures images of the tip and / or peripheral region of the cutlery, and / or a detector 12 that detects tags / codes on the plate can be coupled to the cutlery-type tableware. Accordingly, data can be obtained with the cutlery-type tableware and transmitted to the controller 14 for processing, facilitating the disclosed technology.

[0028] The controller 14 can also include components for facilitating operator interaction with the aspects of the hoodware system 10, such as a display screen and / or input / output device that allows the operator to view the current operating parameters, enter desired operating parameters, and view error logs and past operating parameters. For example, the controller 14 can display information corresponding to any of the data provided to the controller 14, such as the type of food provided to the user, and the current special effects provided to the user, such as the current sound output by the speaker 24. In this way, the operator can monitor the experience and / or provide input for adjusting the user experience. However, it should be understood that the experience can be dynamically controlled and / or fully automated based on data received from the detector 12 and / or other detection devices in the environment (e.g., a camera), such as through indirect input from the user (e.g., via the user's manipulation of the utensils 20 and / or the user's consumption of food and / or via historical data related to the user), and / or without the need for operator selection or input at the controller 14.

[0029] The controller 14 can include a programmable logic controller (PLC) or other suitable control device. The controller 14 can include a processor 16 configured to execute a software program for controlling the speaker 24, the display device 26, and / or the haptic device 28. For example, the processor 16 can control the time and intensity of the sound emitted by the speaker 24, the projector of the light from the projector operating as the display device 26, etc. The processor 16 can process instructions and / or information (e.g., software programs, lookup tables, historical data) stored in the memory device 18. The processor 16 can include a hardware-based processor each including one or more cores. Further, the processor 16 can include a plurality of microprocessors, one or more "general-purpose" microprocessors, one or more system-on-chip (SoC) devices, one or more special-purpose microprocessors, one or more application-specific integrated circuits (ASICs), and / or one or more reduced instruction set computer (RISC) processors. The processor 16 can be communicatively coupled to one or more sensors 30, the speaker 24, the display device 26, and / or the haptic device 28. The controller 14 also includes or is associated with an input / output circuit for receiving data from one or more sensors 30 and an interface circuit for outputting control signals.

[0030] The memory device 18 can include a tangible non-transitory machine-readable medium such as volatile memory (e.g., random access memory [RAM]) and / or non-volatile memory (e.g., read-only memory [ROM], flash memory, hard drive, and / or any other suitable optical, magnetic, or solid-state storage medium). The memory device 18 can store various information for various purposes. For example, the memory device 18 can store machine-readable and / or processor-executable instructions (e.g., firmware or software) for the processor 16 to output special effects in response to data, such as outputting a specific sound in response to a specific movement of the cutlery-type utensils 20 for food. As another example, the memory device 18 can control an actuator coupled to the speaker 24 to move the speaker 24 relative to the user's head and / or calibrate the projector so that a visual image can be easily seen at the position of the utensils 20, the food, and / or the display screen (e.g., tabletop, tablecloth, tray, placemat, plate, charger), and can store instructions for adjusting audio data (e.g., binaural audio) to achieve a desired display by accurately projecting a visual image onto the utensils 20, the food, and / or the display screen.

[0031] FIG. 2 is a schematic diagram showing an embodiment of tableware 20 (e.g., cutlery 20A and plate 20B) that can be used in the hoodware system 10 of FIG. 1. As shown, each identifier 22 can be coupled to an element of the tableware 20 such as the cutlery 20A. The identifier 22 is coupled to the cutlery 20A via a coupler 32 such as a band disposed around the cutlery 20A. However, it should be understood that the identifier 22 can be coupled to the cutlery 20A in other ways (e.g., via an adhesive) and / or integrated (e.g., embedded) within the cutlery 20A. As described herein, the identifier 22 can be removably coupled to the cutlery 20A such that the identifier 22 can be reused with a plurality of different tableware 20 (e.g., of the same type and different types; only of the same type only by being designed to fit around a small handle of the cutlery, etc.). Alternatively, the identifier 22 can remain coupled to the cutlery 20A and be configured such that the identifier 22 is permanently associated with the cutlery 20A over time (e.g., permanently associated with the cutlery 20A in a lookup table).

[0032] Referring to FIGS. 1 and 2, the detector 12 and / or one or more other sensors 30 can collect data indicating the position and / or movement of the cutlery 20A relative to the plate 20B. Further, one or more sensors 30 can collect data indicating the type of tableware 20, the type of food provided to the user, the position of the food, the movement of the food, the position of the user, the movement of the user, and / or the posture of the user. For example, combining the data can indicate that the cutlery 20A is located on a portion of the plate 20B containing food (e.g., pasta).

[0033] One or more sensors 30 can transmit such data to the controller 14. Subsequently, in response to determining that the user is moving the cutlery 20A towards the food item (e.g., within a threshold distance of the food item) and / or that the user has placed the cutlery 20A on the food item (e.g., is in contact with the food item), the controller 14 can instruct the speaker 24 to output a sound effect such as a selected sound effect that represents or mimics laughing in response to being tickled. It should be understood that the selected sound effect can include any of a variety of sounds, including sounds representing screams, laughter, and / or spoken words (e.g., "Please eat me!"). Further, using binaural audio technology, the speaker 24 can output the sound effect such that the user hears and / or interprets the sound effect as being emitted from the food.

[0034] As described herein, the controller 14 can derive any of a variety of information from the data. For example, in response to determining that the user is moving the cutlery 20A (e.g., having at least a portion of the food) away from the plate 20B and / or towards the user's mouth, the controller 14 can control the strength of the audio data output by the speaker 24 (e.g., increase or decrease the volume of the sound). For example, as the cutlery 20A approaches the user's mouth, the speaker 24 can emit a louder sound (e.g., shouting "Please eat!") that appears to be emitted from the food.

[0035] According to one embodiment, the plate 20B can include a coordinate grid and / or can be color-coded (e.g., different colors at different locations) to facilitate the detection of food and / or other parameters via one or more sensors 30. For example, food items can be arranged or plated in a specific known manner such that a first type of food item (e.g., vegetables) is placed in a first portion of the coordinate grid / color on the plate 20B and a second type of food item (e.g., bread) is placed in a second portion of the coordinate grid / color on the plate 20B. This enables the detection of activities related to specific foods and the presentation of effects based thereon. In one embodiment, when it is detected (either via the detection of the identifier 22 or via the detection of the cutlery 20A by one or more other sensors 30) that the cutlery 20A is located (e.g., overlapping) in the first portion, the detector 12 (or one or more other sensors 30) transmits data indicating the position of the cutlery 20A to the controller 14. To facilitate this, multiple detectors 12 can be arranged with respect to the plate (e.g., associated with each quadrant of the plate 20B), although other configurations and techniques are also envisioned. Next, based on the type of food at that location, the controller 14 can select a special effect and command the speaker 24, the display device 26, and / or the haptic device 28 to output the special effect. According to one embodiment, one or more sensors 30 can be triggered to transmit data in this manner (e.g., upon detection of the identifier 22 and / or the cutlery 20A). However, additionally or alternatively, one or more sensors 30 can transmit data to the controller 14 periodically and / or continuously for processing (e.g., during at least the user's dining experience).

[0036] FIG. 3 is a schematic diagram of one embodiment of a dining environment 100 including a foodware system 10. As shown, the dining environment 100 includes tableware 20 and seats 102 (e.g., chairs) associated with respective users 104 (e.g., a first seat 102A with a first user 104A and a second seat 102B with a second user 104B). The dining environment 100 also includes one or more sensors 30, which can include at least a detector 12 and / or a camera 106. As described herein, according to one embodiment, the camera 106 can be considered to be at least one of the detectors 12 that detects an identifier 22 and / or captures an image having the identifier 22. However, the detector 12 may be a reader / scanner different from the camera 106, and the camera 106 may be used to detect other features and / or capture images having other features to facilitate the disclosed technology.

[0037] Each seat 102 may include a number of discrete speakers 24, and the speakers 24 may be of any suitable size or shape. In one embodiment, two speakers, such as speakers 24A, 24B, may be coupled to the seatbacks of respective seats 102A, 102B. Further, seats 102A, 102B can include one or more actuators configured to adjust the respective positions of the speakers 24 (e.g., with respect to user 104). For example, one or more actuators can drive the backing of seat 102 relatively low or high with respect to the underside of seat 102 (e.g., as indicated by arrow 108), thereby adjusting the vertical position of speaker 24 with respect to user 104. In this way, speaker 24 is positioned near the head of user 104. Thus, the binaural audio generated by speaker 24A is customized for first user 104A based on the position of the utensils 20 associated with first user 104A, the movement of the utensils 20 associated with first user 104A, the type of food contacted by the cutlery-type utensils 20 held by first user 104A, the food consumed by first user 104A, the eating habits of first user 104A, etc. Similarly, the binaural audio generated by speaker 24B is customized for second user 104B based on the same or similar parameters (e.g., any of the parameters defined herein). The user (and their positioning) is monitored (e.g., via face recognition or other attribute recognition including anonymous attribute recognition), and the positioning of the speakers for a particular seat can be adjusted to correspond to the user positioned at that particular seat.

[0038] One or more sensors 30 can include a detector 12, a camera 106, a motion sensor, a food weight sensor, a food displacement sensor, and / or other suitable sensors. The one or more sensors 30 can be disposed at any of various positions with respect to the dining environment 100. For example, the detector 12 and / or the camera 106 can be disposed above the tableware 20 (e.g., suspended from the ceiling of the dining environment 100 and / or coupled to the ceiling). As another example, the motion sensor, the food weight sensor, and / or the food displacement sensor can be coupled to the tableware 20. In this way, the motion sensor can detect the movement of the tableware 20 (e.g., cutlery, cup), while the food weight sensor and / or the food displacement sensor can detect changes in the amount and / or position of the food on the tableware 20 (e.g., plate, cup). In response to changes in the amount and / or position of the food, the controller 14 can command specific special effects such as a specific sound via the speaker 24 and / or a projected image onto the food (e.g., a sound is emitted to simulate a wail from the food). Further, the data from the food weight sensor and / or the food displacement sensor can indicate the rearrangement of the food from one plate to another. In response to the rearrangement of the food, the controller 14 can command specific special effects such as a specific sound via the speaker 24 and / or light or an image projected onto the food (e.g., via the projected light or image, the food appears to change color when transferred to another plate). It should be understood that the specific embodiments herein include the identifier 22 on the tableware 20, but the identifier 22 can be located on other articles including food. For example, a barcode can be embedded in frosting, or a QR code can be embedded in a cake topper. Further, the identifier 22 can include a specific color of the food, such as powdered sugar on a cookie or a red strawberry. Thus, the food can be directly detected and / or monitored via the detector 12 and / or the camera 106, which can be coupled to the tableware 20 (e.g., cutlery) and / or at any other suitable location.This makes it possible to easily determine the relative position and / or interaction between the food and the tableware 20 (e.g., cutlery).

[0039] Generally, as described herein, the data collected by one or more sensors 30 can be provided to the controller 14. Next, the controller 14 can be commanded to output sound to the speaker 24 and / or control the display device 26 to project and / or display visual images among other special effects. For example, other special effects can include tactile effects generated by shaking the table and / or seats 102, flowing an air stream to the user 104, emitting an aroma around the user 104, and the like. As another example, other special effects can include illuminating and / or turning off the light emitters around the dining environment 100 (e.g., to darken the dining environment 100). One or more sensors 30 can individually track the tableware 20 associated with each of the users 104, thereby providing a personalized experience to the users 104 (e.g., when the first user 104A touches the first food on his plate 20A, the speaker 24A outputs a laughing sound, however, when the second user 104B touches the first food on his plate 20B, the speaker 24B outputs a comical screaming sound). The personalized experience for the user 104 may also be based on historical data for each of the users 104 such as whether a particular effect was positively received (e.g., the user 104 continued to eat the food) or negatively received (e.g., the user 104 stopped eating the food for at least a threshold period and / or left food on the plate at the end of the meal) during previous dining experiences. It is envisioned that any special effect disclosed herein can be provided in response to any data and / or combination of data (e.g., parameters) disclosed herein.

[0040] In addition to facilitating special effects during the dining experience, it should be understood that data collected by one or more sensors 30 can be used to notify and / or improve operations in the dining environment 100. For example, macaroni & cheese may be a popular food ordered by a user, but data from a food weight sensor (and / or one or more of the other sensors 30 such as camera 106) can indicate that macaroni & cheese is left on the plates of a number of users. In response, the controller 14 can recommend changing the menu and / or cooking the macaroni and cheese in another way and / or what the restaurant can choose to satisfy the user. Further, one or more sensors 30 can include a microphone that captures sounds generated by the user 104 when consuming food. The controller 14 can analyze the sound to determine whether the user 104 enjoyed the food (e.g., via comparison of the sound with sound signatures associated with different emotions / states). According to one embodiment, the data may be input into a machine learning model, and the machine learning model can output recommendations for adjusting food recipes, menu items, etc. based on the user's eating habits and patterns over time.

[0041] FIG. 4 is a schematic view showing an embodiment of a seat 102 provided with a speaker 24 that can be used in the hoodware system 10 of FIG. 1. As described above, binaural audio can provide an immersive experience to the user by localizing sound based on the orientation and / or movement of the user's head. To account for differences in the user's height (e.g., adult vs. child), the speaker 24 can be automatically and / or mechanically moved based on the position and / or orientation of the user's head. For example, the seat 102 can include one or more rails 130 on which a number of individual speakers 24 are arranged. The controller 14 supplies a signal to an actuator, and the actuator drives the speaker 24 to move along each respective rail 130 so that the speaker 24 is appropriately positioned near the user's head to provide binaural audio. For example, the speaker 24 can be positioned at a lower position along each respective rail 130 when the user has a first small height (e.g., when the user is a child). However, the speaker 24 can be positioned at a higher position along each respective rail 130 when the user has a second large height (e.g., when the user is an adult). The user's height may be collected from historical data (which can include, for example, a user profile), input by the user and / or an operator of the dining experience, and / or determined via analysis of data obtained by one or more sensors (e.g., images captured by a camera). According to one embodiment, the user and / or operator can manually adjust the speaker 24 to position the speaker near the user's head.

[0042] With the above in mind, FIG. 5 is a flowchart of one embodiment of a process 150 for generating binaural audio via a headwear system such as the headwear system 10 of FIG. 1. Any suitable device (e.g., processor 16) can execute the process 150. According to one embodiment, the process 150 can be implemented by executing instructions stored in a non-transitory computer-readable medium (e.g., memory device 18). The process 150 is described using blocks in a particular order, but it should be understood that the present disclosure contemplates that the blocks may be executed in any suitable order, that certain blocks may be omitted, and / or that other blocks may be added.

[0043] In block 152, the processor can identify an interaction between food and tableware (e.g., cutlery-type tableware such as a fork, knife, or spoon). For example, the processor can receive data from one or more sensors including indicating an identifier coupled to the tableware, indicating the position of the tableware, indicating the movement of the tableware, and / or indicating the type of food. The interaction can include contact between the food and the tableware, the start and / or continuation of movement of the tableware towards the food (e.g., for a threshold time such as one second), reaching a threshold distance between the food and the tableware, and / or carrying the food towards the user's mouth via the tableware. For example, data from one or more sensors (e.g., a signal from a detector and / or an image captured by a camera) can indicate that the user has inserted a spoon into a soup bowl to contact the soup, that the user is moving a knife towards a slice of bread, and / or that the knife is within a threshold distance of the slice of bread, and / or that the user is carrying a bite of cake towards the user's mouth.

[0044] In block 154, the processor can command the speaker to provide an audible output in response to an interaction. According to one embodiment, the processor can instruct the speaker to provide binaural audio such that the audio data appears to emanate from the food. For example, when the user touches a cake, the speaker can output a sound to provide the effect that the cake is singing "Happy Birthday". The processor can also control one or more actuators that move the speaker, as described herein. The speaker can be placed on the seat where the user is sitting and / or in other appropriate locations related to the dining environment.

[0045] In block 156, the processor can command one or more display devices to provide a visual output in response to an interaction. In one embodiment, the processor can instruct the display device to project an image on the food and / or in the vicinity of the food (e.g., on the plate supporting the food, on the cup holding the food, on the table supporting the plate or cup having the food, on the user or another user at the table). For example, when the user touches a cake, the display device can project an image of a monster on the cake. The processor can also adjust the sound from the speaker and the image from the display device. For example, the sound can include phrases and / or laughter, and the image can include a monster moving its mouth substantially in synchronization with the phrases and / or laughter. The display device can be placed on the ceiling and / or in other appropriate locations within the dining environment.

[0046] In block 158, the processor can instruct one or more haptic devices to provide a haptic output in response to an interaction. For example, the processor can instruct the haptic device to shake a table that supports food, direct an air flow towards the user, emit an aroma in the vicinity of the user, or the like. The processor can also adjust the sound from a speaker, the image from a display device, and / or the haptic output from a haptic device. For example, the image can include a flame, and the haptic output can include a warm air flow directed towards the user substantially in synchronization with the flame. The haptic device can be disposed at any suitable location on a table, a seat, tableware, a ceiling, a wall, and / or any other suitable location in the dining environment. In fact, one or more sensors, speakers, display devices, and / or haptic devices according to one embodiment may be coupled to a table, a seat, a ceiling, a wall, tableware (e.g., cutlery-type tableware, plates, cups), and / or any other suitable location in the dining environment.

[0047] It should be understood that the foodware system can provide any of various interactive special effects in response to any of various events in a dining environment. For example, the processor can track the movement of the plate before it is presented to the user and / or when the plate is provided to the user. Based on determining that the plate has been handed to the user (e.g., set on the table), the processor can instruct the speaker and / or the haptic device to generate an auditory stimulus and a tactile stimulus, respectively. For example, the user can hear a thumping sound effect and feel the seat shake in response to and / or concomitant with the plate being set on the table. According to one embodiment, the processor can utilize binaural audio, visual images, and / or tactile stimuli to create a storyline during the dining experience. Different courses of the dining experience (e.g., appetizer, main course, dessert) can correspond to different parts of the storyline. Other variations are also envisioned. For example, the foodware system can be configured to more generally identify interactions between the food and the user via the user's hand (e.g., the user pinching a cookie), via other movements (e.g., chewing movements), and / or via tableware (e.g., the user moving a cup to drink a beverage, the user moving a knife to cut bread). In response thereto, the foodware system can provide any of various special effects, such as any of the special effects disclosed herein.

[0048] Although only specific features of the invention have been illustrated and described herein, many modifications and variations will occur to those skilled in the art. Accordingly, it should be understood that the appended claims are intended to cover all such modifications and variations that fall within the true technical spirit of the present disclosure.

[0049] The methods presented and claimed in this specification are referenced and applied to the substantial objects and specific examples of a practical nature that clearly improve the technical field of the present invention, and for this reason, are not abstract, intangible, or truly theoretical. Further, if any of the claims appended at the end of this specification include one or more elements designated as "means for [performing a function]" or "steps for [performing a function]", such elements shall be construed in accordance with 35 U.S.C. § 112(f). However, for any claim that includes elements designated in any other way, such elements shall not be construed in accordance with 35 U.S.C. § 112(f).

Explanation of Signs

[0050] 30 Sensor 12 Detector 14 Controller 20 Tableware 22 Identifier 24 Speaker 26 Display Device 28 Haptic Device

Claims

1. A headwear system comprising: an identifier coupled to an instrument configured to be used by a user to engage food; a detector configured to generate data indicative of the position of the instrument, the movement of the instrument, or both, based on detection of the identifier; a controller communicatively coupled to the detector and configured to command a speaker to generate audio data based on the position of the instrument, the movement of the instrument, or both; and a headwear system.

2. The headwear system according to claim 1, wherein the audio data includes binaural audio data providing an effect of the audio data as if it were emitted from the food.

3. The headwear system according to claim 1, wherein the identifier includes a radio frequency identification (RFID) tag, a near field communication (NFC) tag, a barcode, a quick response (QR) code, or any combination thereof.

4. The headwear system according to claim 1, wherein the detector includes a radio frequency identification (RFID) reader, a near field communication (NFC) reader, a barcode scanner, a quick response (QR) code scanner, or any combination thereof.

5. The headwear system according to claim 1, further comprising a camera configured to capture an image representing the position of the instrument, the movement of the instrument, the movement of the user, the type of food, the position of the food, the movement of the food, or any combination thereof.

6. a food displacement sensor configured to detect movement of the food; a weight sensor configured to detect the weight of the food; a motion sensor configured to detect the movement of the user, the movement of the instrument, or both; or any combination thereof, The headwear system according to claim 1.

7. The headwear system according to claim 1, further comprising the speaker coupled to a chair configured to support the user, wherein the controller is configured to move the speaker relative to the position of the head of the user by an actuator.

8. The hoodware system according to claim 1, wherein the controller is configured to dynamically project an image onto the food, the tableware supporting the food, or both based on the position of the appliance, the movement of the appliance, or both.

9. The hoodware system according to claim 1, wherein the controller is configured to command a haptic device to provide a tactile stimulus based on the position of the appliance, the movement of the appliance, or both.

10. The hoodware system according to claim 1, wherein the controller is configured to command the speaker to generate the audio data indicating contact between the appliance and the food in response to the position of the appliance, the movement of the appliance, or both.

11. The hoodware system according to claim 1, wherein the detector is coupled to a plate supporting the food.

12. A method for providing an immersive dining experience, comprising: identifying an interaction between food and a user based on data obtained by one or more sensors via one or more processors; projecting an image onto the food in response to the interaction between the food and the user via a projector; The method includes the above steps.

13. The method according to claim 12, further comprising outputting audio data in response to the interaction via a speaker.

14. The method according to claim 12, further comprising outputting a tactile stimulus in response to the interaction via a haptic device.

15. The step of identifying the interaction between the food and the user includes identifying contact between the food and an appliance, according to the method of claim 12.

16. The step of identifying the interaction between the food and the user includes identifying the start of movement of an appliance held by the user towards the food, the continuous movement of the appliance towards the food over a threshold time, the appliance being within a threshold distance of the food, and / or the food being moved towards the user's mouth via the appliance, or any combination thereof, according to the method of claim 12.

17. The method according to claim 12, wherein the data includes an image captured by a camera, proximity data obtained by one or more proximity sensors, relative position data obtained by one or more radio frequency identification (RFID) readers, or any combination thereof.

18. A headwear system, comprising: one or more sensors configured to obtain data indicating an interaction between food and a user; a controller communicatively coupled to the one or more sensors and configured to instruct a projector to project an image onto the food in response to the interaction between the food and the user; The headwear system comprising the above.

19. The headwear system according to claim 18, wherein the controller is configured to instruct a speaker to generate audio data in response to the interaction between the food and the user.

20. The headwear system according to claim 18, wherein the controller is configured to select the image based on the type of the food, historical data related to the user, or both.

Images