Interactive food system and method

Abstract

A foodstuff system (10) includes an identifier (22) coupled to an appliance (20) configured to be used by a user (104) to engage a foodstuff. The feeder system (10) further comprises a detector (12) configured to generate data indicative of a position of the appliance (20), a movement of the appliance (20), or both, based on the detection of the identifier (22). The feeder system (10) also includes a controller (14) communicatively coupled to the detector (12) and configured to instruct the speaker (24) to generate audio data based on the position of the appliance (20), the movement of the appliance (20), or both.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 341,308, filed on May 12, 2022, entitled “INTERACTIVE FOODWARE SYSTEMS AND ETHODS,” which is hereby incorporated by reference in its entirety for all purposes. Background Art

[0002] This section is intended to introduce the reader to various aspects of the technology that may be related to various aspects of the technology. This discussion is considered to be helpful in providing the reader with background information in order to promote a better understanding of the various aspects of the present disclosure. Therefore, it should be understood that these statements are to be read from this perspective rather than as an admission of the prior art.

[0003] Amusement parks generally include attractions that provide users with various experiences. For example, an amusement park may include different attractions, such as roller coasters, drop towers, log slides, etc. Some attractions may include environments that provide effects that help provide users with an immersive experience, such as auditory stimulation, tactile stimulation, visual stimulation, and / or other special effects. Amusement parks may include other types of attractions and / or venues, such as restaurants that provide users with dining experiences. Summary of the invention

[0004] Certain embodiments commensurate in scope with the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of the present disclosure, but rather these embodiments are intended only to provide a brief overview of certain disclosed embodiments. In fact, the present disclosure may encompass a wide variety of forms that may be similar or different from the embodiments set forth below.

[0005] In one embodiment, a food utensil system includes an identifier coupled to a utensil configured to be used by a user to engage food. The food utensil system also includes a detector configured to generate data indicating a position of the utensil, movement of the utensil, or both based on detection of the identifier. The food utensil system also includes a controller communicatively coupled to the detector and configured to instruct a speaker to generate audio data based on the position of the utensil, movement of the utensil, or both.

[0006] In one embodiment, a method for providing an immersive dining experience includes identifying, via one or more processors and based on data obtained by one or more sensors, an interaction between a food product and a user. The method also includes projecting an image onto the food product via a projector in response to the interaction between the food product and the user.

[0007] In one embodiment, a food processor system includes one or more sensors configured to obtain data indicating an interaction between a food product and a user. The food processor system also includes a controller communicatively coupled to the one or more sensors and configured to instruct a projector to project an image onto the food product in response to the interaction between the food product and the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which like characters represent like parts throughout the drawings, and in which:

[0009] Figure 1 is a block diagram of a food ware system for generating auditory, tactile and / or visual stimulation during a dining experience according to an embodiment of the present disclosure;

[0010] Figure 2 According to an embodiment of the present disclosure, Figure 1 Schematic illustration of tableware used in a food utensil system;

[0011] Figure 3 According to an embodiment of the present disclosure, a device for use in a dining environment Figure 1 Schematic illustration of the feeding system;

[0012] Figure 4 According to an embodiment of the present disclosure, Figure 1 A schematic illustration of a user seat used in a food container system of; and

[0013] Figure 5 According to an embodiment of the present disclosure, Figure 1 Flowchart of the process by which the predator system generates auditory stimuli. DETAILED DESCRIPTION

[0014] One or more specific embodiments of the present disclosure will be described below. In order to provide a concise description of these embodiments, not all features of an actual implementation may be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. In addition, it should be appreciated that such development work may be complex and time-consuming, but for an ordinary technician who benefits from the present disclosure, it will be nothing more than a routine task of design, fabrication, and manufacturing.

[0015] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements described. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements besides the listed elements. Additionally, it should be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be interpreted to exclude the existence of additional embodiments that also incorporate the recited features.

[0016] The present disclosure generally relates to the field of special effects for interactive environments (such as amusement parks). More specifically, the present disclosure relates to systems and methods for generating auditory stimulation (e.g., binaural audio), tactile stimulation, and / or visual stimulation during a dining experience. As used herein, binaural audio can create a three-dimensional (3D) stereoscopic sense for a user based on how the human auditory system receives and processes sound. For example, if a dog barks at the user's left ear, the barking sound may take more seconds to reach the user's right ear than the left ear. The barking sound may also be louder in one ear than in the other ear. Binaural audio takes into account such differences in time and intensity associated with the sound. By localizing the sound based on the orientation and / or movement of the human head, binaural audio can provide an immersive experience to the user. For example, a user can hear a whisper in one ear or hear a bird flying overhead.

[0017] In addition, tactile stimulation can create a sense of touch for the user, such as by applying force, vibration and / or motion to the user. As used herein, tactile stimulation can include smell (e.g., olfactory sense). In addition, visual stimulation can include a projected image (e.g., two-dimensional [2D] or 3D image) that can be observed by the user (e.g., with or without viewing glasses). In one embodiment, projection mapping technology can be used to generate the projected image and project the projected image onto tableware (e.g., plate, tabletop) and / or (one or more) food during the dining experience (e.g., to create a virtual enhancement of tableware and / or (one or more) food and / or a superimposed image of virtual features on tableware and / or (one or more) food). Projection mapping technology can be used additionally or alternatively to project the projected image onto any other type of display surface (such as a traditional movie screen). 2D images refer to things that are typically considered to be "flat" images that appear to exist in two dimensions when viewed by a user, and 3D images are also "flat" images that are provided in a manner that appears to exist in three dimensions when viewed by a user. Both 2D images and 3D images may be provided on non-planar surfaces (such as food, textured tablecloths, etc.).

[0018] In view of the foregoing, an amusement park or other venue may include a foodware system (e.g., an interactive foodware system) configured to create special effects by generating auditory, visual, and / or tactile stimulation during a dining experience. In fact, a combination of certain hardware configurations (e.g., circuitry), software configurations (e.g., algorithmic structures and / or modeled responses), and certain attraction features may be utilized to provide an immersive dining experience for users.

[0019] The tableware system may include tableware with a sensing component (e.g., a sensor). Additionally or alternatively, the sensing component may be distributed around the dining environment (e.g., at a position separated from the tableware). In addition, the tableware system may include a speaker for generating auditory stimulation (e.g., binaural audio), a tactile device for generating tactile stimulation, and / or a projector or other display device for generating visual stimulation based on the data obtained by the sensing component. The tableware may include a plate, a bowl, a pan, a container, a cup, a utensil / cutlery, a desktop, a napkin, etc. The tableware system may track various parameters of tableware, (one or more) food, and / or (one or more) users. For example, the tableware system may track food (e.g., edible food, including solid food and / or liquid food (such as water and other beverages)), such as the movement of food (e.g., the movement of food relative to tableware, wherein the movement of food is caused or given by the user (such as via cutlery)). The food utensil system can track the tableware, including the movement of the tableware (e.g., the movement of the cutlery relative to the food, wherein the movement of the cutlery is caused or imparted by the user). The food utensil system can then provide auditory, visual and / or tactile stimulation based on the movement of the food and / or the movement of the tableware. The food utensil system can determine the user's eating habits and patterns (e.g., the type of (one or more) food supplied to and / or consumed by the user, the amount of each type of food supplied to and / or consumed by the user, the amount of each type of food supplied to and / or consumed by the user, the user's consumption rate of each type of food, the order of consumption of different types of food when presented to the user, and / or the sound made by the user when eating each type of food). In such cases, the food utensil system can provide auditory, visual and / or tactile stimulation based on the user's eating habits and patterns.

[0020] As a specific example of facilitating discussion, in response to determining that a user is manipulating a knife to slice a loaf of bread on a plate, the food processor system may generate binaural audio that creates the effect of a sizzling sound that appears to the user to originate from the loaf of bread and visual imagery that depicts what appears to the user to be a snake slithering from the loaf of bread onto the plate. It should be appreciated that the food processor system also temporally coordinates the auditory, visual, and / or tactile stimuli with each other and with detected movement (e.g., movement of cutlery and / or food, such as movement of a knife and bread) to provide an overall immersive effect for the user.

[0021] To generate binaural audio and other effects in this manner, the controller of the food utensil system receives data from various sensing components (e.g., detectors and other sensors). Non-limiting examples of detectors may include radio frequency identification (RFID) readers, near field communication (NFC) readers, barcode scanners, quick response (QR) code scanners, and / or cameras. The detector may be communicatively coupled to the controller of the food utensil system. Similarly, non-limiting examples of identifiers detectable by the detector may include RFID tags, NFC tags, barcodes, and QR codes. Some additional non-limiting examples of identifiers detectable by the detector may include visual patterns, retroreflective materials with specific colors (e.g., visible or invisible to humans), shapes of materials or items (e.g., along unique edges of tableware items). In one embodiment, the corresponding identifier may be coupled to the corresponding tableware and / or integrated into the corresponding tableware. For example, an identifier (such as an RFID tag) may be coupled to a knife for slicing a loaf of bread. The detector (such as an RFID reader) may identify the position and / or movement of the knife based on the detection of the identifier (e.g., communication therewith). The detector may then send position and / or movement data associated with the knife to the controller. The controller can process the position and / or movement data associated with the knife to determine that the movement of the knife corresponds to a cutting or slicing motion (e.g., by referencing a library of stored motions and / or classifying the movement of the knife via one or more algorithms). The controller can then instruct the speakers to generate binaural audio, the projector to project visual images, etc., as noted herein, in order to create an immersive dining experience for the user.

[0022] The controller may process data generated by and received from the detector(s) and / or other sensors to track users (e.g., actions of users), food(s) (e.g., type of food(s), location of food(s), displacement of food(s), weight of food(s)), and / or cutlery (e.g., location of cutlery(s), movement of cutlery(s). For example, the controller may utilize any of a variety of tracking techniques and tools, such as RFID monitoring, NFC monitoring, proximity sensor(s), coordinate grid(s), camera(s), motion sensor(s), food displacement sensor(s), food weight sensor(s), color marking(s), computer vision, and the like. Based on data related to the user, food(s), cutlery, and dining environment, the controller may effectively generate responsive and / or interactive auditory, tactile, and / or visual stimuli during the dining experience. In addition, the controller may transmit the data and / or related information to a computing device, server, remote server, and the like associated with a restaurant or dining attraction personnel. The data and / or related information can help restaurants prepare and customize food based on the user's eating habits and patterns. For example, in addition to tracking the type of food(s) ordered, restaurants can also gain insights from the type of food(s) ordered but not consumed by the user (e.g., food left on the plate).

[0023] Go to the attached picture, Figure 1 A block diagram of one embodiment of an eating utensils system 10 (e.g., an interactive eating utensils system) is shown. As shown, the eating utensils system 10 may include (one or more) detectors 12, a controller 14 (e.g., a programmable logic controller or a computer), tableware 20, (one or more) speakers 24, (one or more) display devices 26, and / or (one or more) tactile devices 28. It should be appreciated that the eating utensils system 10 may omit Figure 1 Some of the elements shown in and / or may include additional elements, such as lighting elements (e.g., light emitters) that provide special lighting effects. The controller 14 may include (one or more) processors 16 and (one or more) memory devices 18. In addition, each item of tableware 20 may be coupled to a corresponding identifier 22.

[0024] The detector(s) 12, the tableware 20 (e.g., including the corresponding identifier(s) 22), the speaker(s) 24, the display device(s) 26, and / or the tactile device(s) 28 are communicatively coupled to the controller 14. The detector(s) 12 may include any suitable type of detector configured to detect the corresponding identifier(s) 22. More specifically, the detector(s) 12 may include any suitable type of detector configured to generate data (e.g., a signal) indicating the position and / or movement of the tableware 20 via detection of the corresponding identifier(s) 22. For example, the detector(s) 12 may include a radio frequency identification (RFID) reader, a near field communication (NFC) reader, a barcode scanner, a quick response (QR) code scanner, a camera, or any combination thereof. In such cases, the identifier(s) 22 may include a corresponding detectable or readable identifier, such as an RFID tag, an NFC tag, a barcode, and a QR code, or any combination thereof. Additionally or alternatively, the identifier(s) 22 may include a visual pattern, a retroreflective material having a particular color (eg, visible or invisible to humans), and / or a shape of a material or item (eg, along a unique edge of a tableware item).

[0025] Thus, in operation, one identifier 22 may be integrated into and / or coupled to a first piece of cutlery 20 (e.g., a fork), and / or another identifier 22 may be integrated into and / or coupled to a second piece of cutlery 20 (e.g., a plate), and / or the like. The detector(s) 12 may then generate data indicating the respective positions and / or respective movements of the first piece of cutlery 20 and / or the second piece of cutlery 20 via detection of the identifier(s) 22. In particular, the detector(s) 12 may generate data indicating the positions and / or movements of the first piece of cutlery 20 and the second piece of cutlery 20 by periodically and / or continuously detecting and tracking the identifier(s) 22 (e.g., in a three-dimensional [3D] space within an environment such as a performance environment or a dining environment).

[0026] The detector(s) 12 may provide data indicating the position and / or movement of the tableware 20 to the controller 14 during an experience (e.g., a performance experience or a dining experience), such as periodically and / or continuously. The controller 14 may execute instructions to process the data to determine the position and / or movement of the tableware 20. In one embodiment, the controller 14 may reference one or more databases (e.g., a lookup table) to identify a particular piece of tableware 20 associated with the identifier 22. In addition, the controller 14 may also reference one or more databases to identify a user who is at least currently associated with the particular piece of tableware 20 and / or the identifier 22. For example, a user may be associated with the identifier 22 when purchasing the identifier 22 (e.g., a band), and then the user may reuse the identifier 22 with different pieces of tableware 20 over time (e.g., attached to a fork during one experience, and then attached to another fork during another experience). As another example, a user may be associated with the piece of tableware 20 and the identifier 22 of the piece of tableware 20 (e.g., integrated into the piece of tableware 20 or otherwise affixed to the piece of tableware 20) when purchasing the tableware 20, and the user may then reuse the piece of tableware 20 over time. As yet another example, the identifier 22 may be integrated into the piece of tableware 20 or otherwise reused with the piece of tableware 20, where the piece of tableware 20 is reused by multiple different users over time. In such cases, the piece of tableware 20 and the identifier 22 of the piece of tableware 20 may be temporarily assigned to or associated with the user during the experience (e.g., when sitting down at the table).

[0027] In this way, the controller 14 can access information related to the characteristics of the particular piece of tableware 20 (e.g., type, such as a knife or plate) and / or historical data related to the user's previous experience (e.g., the user reacted positively to certain special effects). However, it should be appreciated that the techniques disclosed herein can be implemented without associating the tableware 20, identifiers 22, and / or users with each other. For example, any identifier 22 within the detectable range of the detector(s) 12, and / or other sensors can be utilized, and the controller 14 can then control other elements accordingly.

[0028] In one embodiment, the controller 14 may execute instructions to control the speaker(s) 24 to generate audio data (e.g., binaural audio), control the display(s) 26 to generate visual images, and / or control the haptic device(s) 28 to generate tactile output based on data received from the detector(s) 12 indicating the identifier 22, the position of the tableware 20, and / or the movement of the tableware 20. In addition, the controller 14 may receive and process other types of data to determine when and how to operate the speaker(s) 24, the display(s) 26, and / or the haptic device(s) 28. For example, the controller 14 may receive and process data indicating the type of tableware 20, the type of food(s) served to the user, the position of the food(s) (e.g., the food position of the food(s)), the movement of the food(s) (e.g., the food movement of the food), the position of the user (e.g., the user position of the user), the movement of the user (e.g., the user movement of the user), and / or the body position of the user (e.g., the orientation of the user's head). The controller 14 may also derive other information from the data, such as the relative position of (one or more) foods and cutlery 20 (e.g., movement of the cutlery 20 relative to a particular food) and / or the interaction between (one or more) foods and cutlery 20 (e.g., contact between a particular food and cutlery 20).

[0029] As shown, the food utensil system 10 may include (one or more) detectors 12 configured to detect the identifier 22 (e.g., via radio frequency communication, code reading technology and / or image processing technology) and one or more other sensors 30 that operate to collect other types of data. One or more sensors 30 may include (one or more) cameras, (one or more) motion sensors, (one or more) food weight sensors, (one or more) food displacement sensors or any combination thereof. It should be appreciated that one or more sensors 30 may collect the data described herein via any suitable technology. For example, (one or more) detectors 12 may be configured to detect the identifier 22 (e.g., obtain a unique identifier of an RFID tag or other code), and the identifier 22 may be able to achieve association with a particular tableware 20 and / or user. (One or more) detectors 12 may also continuously detect the identifier 22 over time to effectively track the position and / or movement of the identifier 22 and its tableware 20. Additionally or alternatively, the camera(s) (which may be considered as or operate as the detector(s) 12 for the identifier 22, and / or may operate as an addition to the detector(s) 12 for tracking the tag or other code identifier 22 via techniques other than image processing techniques) may track the position and / or movement of the tableware 20. For example, the camera(s) may capture an image of the environment and provide the image of the environment to the controller 14. The controller 14 may then apply computer vision techniques (e.g., template matching, object recognition, gesture recognition) to track the position and / or movement of the tableware 20. In fact, analysis of the images may be used to determine any of a wide variety of data, including the type of tableware 20, the type of food(s) served to the user, the location of the food(s), the movement of the food(s), the location of the user, the movement of the user, and / or the body position of the user. Furthermore, it should be appreciated that the detector(s) 12 may be omitted, and / or any of the one or more sensors 30 may be used alone or in any combination to collect any of the data described herein.

[0030] It should also be appreciated that the one or more sensors 30 may include one or more proximity sensors (e.g., capacitive sensors) located in the tableware 20. The one or more proximity sensors may be configured to detect an object, such as via a detectable tag (e.g., a metal tag, a magnet) embedded in the object. As an example, one or more proximity sensors may be positioned on a tabletop and may be configured to detect a food container (e.g., a plate or a cup) when the food container is within the range of the one or more proximity sensors. As another example, one or more proximity sensors may be positioned on a plate and may be configured to detect cutlery (e.g., utensils) when the cutlery is within the range of the one or more proximity sensors. In such cases, the data obtained by the one or more proximity sensors indicates the position and / or movement of one item relative to another item (e.g., cutlery relative to a plate). The one or more proximity sensors may transmit data to the controller 14 for processing and facilitate the disclosed technology.

[0031] It should be appreciated that a plurality of proximity sensors may be positioned around the dish, such as a first proximity sensor at the first quadrant of the dish and a second proximity sensor at the second quadrant of the dish. In addition, it may be known (e.g., via an operator and / or via input to the controller 14 of analysis of images captured by (one or more) cameras) that the first quadrant contains a first food and the second quadrant contains a second food. Then, upon detection of cutlery by the first proximity sensor, the controller 14 may initiate (one or more) first special effects, such as laughter and images of hyenas. Then, upon detection of cutlery by the second proximity sensor, the controller 14 may initiate (one or more) second special effects, such as images of dragons and shaking tactile stimulation. In this way, (one or more) special effects may be responsive to the position and / or movement of cutlery-type tableware, the interaction between different pieces of tableware 20, the interaction between tableware 20 and food, and the type of food. It should be appreciated that one or more sensors 30 may be arranged in other ways, such as on cutlery-type tableware. For example, proximity sensor(s) that detect features of the tray, cameras that capture images of the tip of the cutlery and / or surrounding area, and / or detector(s) 12 that detect labels / codes on the tray may be coupled to the cutlery-type tableware. Thus, the data may be obtained at the cutlery-type tableware and transmitted to the controller 14 for processing and facilitating the disclosed technology.

[0032] The controller 14 may also include components to facilitate operator interaction with aspects of the food system 10, such as a display screen and / or input / output device that enables the operator to check current operating parameters, input desired operating parameters, check error logs and historical operating parameters, etc. For example, the controller 14 may display information corresponding to any of the data provided to the controller 14, such as the type of (one or more) food supplied to the user and the current special effects provided to the user (such as the current sound output by (one or more) speakers 24). In this way, the operator can monitor the experience and / or provide input to adjust the user's experience. However, it should be appreciated that the experience can be dynamically controlled and / or fully automated (e.g., only via indirect input from the user (such as via the user's manipulation of the tableware 20 and / or the user's consumption of (one or more) food and / or historical data related to the user), and / or without any selection or input by the operator at the controller 14).

[0033] The controller 14 may include a programmable logic controller (PLC) or other suitable control device. The controller 14 may include (one or more) processors 16, which are configured to execute software programs to control (one or more) speakers 24, (one or more) display devices 26 and / or (one or more) tactile devices 28. For example, the (one or more) processors 16 may control the timing and intensity of the sound emitted by the (one or more) speakers 24, the projection of light from (one or more) projectors operating as (one or more) display devices 26, etc. The (one or more) processors 16 may process instructions and / or information (e.g., software programs, lookup tables, historical data) stored in (one or more) memory devices 18. The (one or more) processors 16 may include (one or more) hardware-based processors each including one or more cores. In addition, the (one or more) processors 16 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more system-on-chip (SoC) devices, one or more special-purpose microprocessors, one or more application-specific integrated circuits (ASICs) and / or one or more reduced instruction set computer (RISC) processors. Processor(s) 16 are communicatively coupled to one or more sensors 30, speaker(s) 24, display(s) 26, and / or haptic device 28. Controller 14 also includes or is associated with input / output circuitry for receiving data from one or more sensors 30 and interface circuitry for outputting control signals.

[0034] The memory device(s) 18 may include tangible, non-transitory machine-readable media, such as volatile memory (e.g., random access memory [RAM]) and / or non-volatile memory (e.g., read-only memory [ROM], flash memory, a hard drive, and / or any other suitable optical, magnetic, or solid-state storage media). The memory device(s) 18 may store a wide variety of information that may be used for a variety of purposes. For example, the memory device(s) 18 may store machine-readable and / or processor-executable instructions (e.g., firmware or software) for execution by the processor(s) 16 to output a special effect in response to data, such as outputting a particular sound in response to a particular movement of a piece of cutlery-type tableware 20 relative to, for example, the food(s). As another example, the memory device(s) 18 may store instructions that cause the processor(s) 16 to adjust the audio data (e.g., binaural audio) to achieve a desired presentation (e.g., by controlling an actuator coupled to the speaker(s) 24 to move the speaker(s) 24 relative to the user's head, and / or by calibrating a projector to accurately project a visual image onto the tableware 20, the food(s), and / or a display screen (e.g., a table surface, tablecloth, tray, placemat, plate, charger) such that the visual image is easily visible at the location of the tableware 20, the food(s), and / or the display screen).

[0035] Figure 2 is available in Figure 1 Schematic illustration of an embodiment of tableware 20 (e.g., cutlery 20A and plate 20B) used in a food utensil system 10. As illustrated, a corresponding identifier 22 may be coupled to a piece of tableware 20, such as to the cutlery 20A. The identifier 22 is coupled to the cutlery 20A via a coupler 32 (such as a belt) disposed around the cutlery 20A. However, it should be appreciated that the identifier 22 may be coupled to the cutlery 20A in other ways (e.g., via an adhesive) and / or integrated into the cutlery 20A (e.g., embedded therein). As noted herein, the identifier 22 may be removably coupled to the cutlery 20A so that the identifier 22 can be reused with different pieces of tableware 20 (e.g., of the same type and of different types; of the same type, such as of the same type only via a small handle designed to be mounted around the cutlery). Alternatively, the identifier 22 may be configured to remain coupled to the cutlery 20A such that the identifier 22 is associated only with the cutlery 20A over time (eg, permanently associated with the cutlery 20A in a lookup table).

[0036] refer to Figure 1 and 2, the detector(s) 12 and / or one or more other sensors 30 may collect data indicating the position and / or movement of the cutlery 20A relative to the plate 20B. In addition, the one or more sensors 30 may collect data indicating the type of cutlery 20, the type of food(s) served to the user, the position of the food(s), the movement of the food(s), the position of the user, the movement of the user, and / or the body position of the user. For example, the data may together indicate that the cutlery 20A is located on a portion of the plate 20B that includes food (e.g., pasta).

[0037] One or more sensors 30 may send such data to the controller 14. Then, in response to determining that the user is moving the cutlery 20A toward the food (e.g., within a threshold distance of the food) and / or the user has placed the cutlery 20A on the food (e.g., in contact with the food), the controller 14 may instruct the speaker(s) 24 to output a sound effect, such as a selected sound effect in response to an amused expression or imitation of laughter. It should be appreciated that the selected sound effect may include any of a wide variety of sounds, including sounds representing screams, laughter, and / or spoken phrases (e.g., "Eat me!"). In addition, using binaural audio technology, the speaker(s) 24 may output the sound effect in a manner that causes the user to hear and / or interpret the sound effect as originating from the food.

[0038] As noted herein, controller 14 may derive any of a wide variety of information from the data. For example, in response to determining that the user is moving cutlery 20A (e.g., with at least a portion of a food item) away from plate 20B and / or toward the user's mouth, controller 14 may control the intensity of audio data output by speaker(s) 24 (e.g., increase or decrease the volume of the sound). For example, when cutlery 20A approaches the user's mouth, speaker(s) 24 may emit a louder sound that appears to originate from the food item (e.g., shouting "Eat me!").

[0039] In one embodiment, the tray 20B may include a coordinate grid and / or may be color-coded (e.g., different colors in different areas) to facilitate detection of the food(s) and / or other parameters via the one or more sensors 30. For example, the food(s) may be arranged or plated in a particular known manner, such as where a first type of food (e.g., vegetables) is in a first portion of the coordinate grid / colors on the tray 20B, a second type of food (e.g., bread) is in a second portion of the coordinate grid / colors on the tray 20B, and so on. This allows for detection of activity relative to specific food items and for effects to be presented based thereon. In one embodiment, upon detection of the cutlery 20A at (e.g., overlapping) the first portion via detection of the identifier 22 (or via detection of the cutlery 20A with one or more other sensors 30), the detector(s) 12 (or one or more other sensors 30) sends data indicating the location of the cutlery 20A to the controller 14. To facilitate this, multiple detectors 12 may be positioned around the tray (e.g., associated with respective quadrants of the tray 20B)—although other configurations and techniques are contemplated. Then, based on the type of food at that location, controller 14 may select a special effect and instruct speaker(s) 24, display(s) 26, and / or haptic(s) 28 to output the special effect. In one embodiment, one or more sensors 30 may be triggered to transmit data in this manner (e.g., upon detection of identifier 22 and / or cutlery 20A). However, additionally or alternatively, one or more sensors 30 may periodically and / or continuously transmit data to controller 14 for processing (e.g., at least during the user's dining experience).

[0040] Figure 3 is a schematic illustration of an embodiment of a dining environment 100 including a food utensil system 10. As shown, the dining environment 100 includes food utensil 20 and seats 102 (e.g., chairs) associated with respective users 104 (e.g., a first seat 102A with a first user 104A and a second seat 102B with a second user 104B). The dining environment 100 also includes one or more sensors 30, which may include at least (one or more) detectors 12 and / or (one or more) cameras 106. As noted herein, in one embodiment, the (one or more) cameras 106 may be considered to be at least one of the (one or more) detectors 12 that detects an identifier 22 and / or captures an image with the identifier 22. However, the (one or more) detectors 12 may be readers / scanners that are different from the (one or more) cameras 106, and the (one or more) cameras 106 may be used to detect other features and / or capture images with other features in order to facilitate the disclosed technology.

[0041] Each seat 102 may include several discrete speakers 24, and (one or more) speakers 24 may be of any suitable size or shape. In one embodiment, two speakers (such as speakers 24A and 24B) may be coupled to the backs of the respective seats 102A and 102B. In addition, seats 102A and 102B may include one or more actuators configured to adjust the respective positions of (one or more) speakers 24 (e.g., relative to user 104). For example, one or more actuators may drive the back of the seat 102 to be lower or higher (e.g., as shown by arrow 108) relative to the bottom of the seat 102, so as to thereby adjust the vertical position of (one or more) speakers 24 relative to the user 104. In this way, (one or more) speakers 24 may be positioned near the head of the user 104. As such, binaural audio generated by the speaker(s) 24A is customized for the first user 104A based on the location of the cutlery 20 associated with the first user 104A, the movement of the cutlery 20 associated with the first user 104A, the type of food(s) contacted by the cutlery-type cutlery 20 held by the first user 104A, the food(s) consumed by the first user 104A, the eating habits of the first user 104A, etc. Similarly, binaural audio generated by the speaker(s) 24B is customized for the second user 104B based on the same or similar parameters (e.g., any of the parameters set forth herein). Users (and their positioning) may be monitored (e.g., via facial recognition or other attribute recognition, including anonymous attribute recognition), and the positioning of the speakers for a particular seat may be adjusted to accommodate the user positioned in the particular seat.

[0042] The one or more sensors 30 may include detector(s) 12, camera(s) 106, motion sensor(s), food weight sensor(s), food displacement sensor(s), and / or other suitable sensors. The one or more sensors 30 may be positioned at any of a variety of locations around the dining environment 100. For example, the detector(s) 12 and / or camera(s) 106 may be positioned above the tableware 20 (e.g., suspended from and / or coupled to a ceiling of the dining environment 100). As another example, the motion sensor(s), food weight sensor(s), and / or food displacement sensor(s) may be coupled to the tableware 20. In this manner, the motion sensor(s) may detect movement of the tableware 20 (e.g., cutlery, cups), while the food weight sensor(s) and / or food displacement sensor(s) may detect changes in the amount and / or position of the food(s) on the tableware 20 (e.g., plates, cups). In response to any change in the amount and / or position of (one or more) foods, the controller 14 may indicate certain special effects, such as certain sounds via (one or more) speakers 24 and / or projected images onto (one or more) foods (e.g., sounds are emitted to simulate screams from (one or more) foods). In addition, data from (one or more) food weight sensors and / or (one or more) food displacement sensors may indicate the migration of (one or more) foods from one plate to another. In response to the migration of (one or more) foods, the controller 14 may indicate certain special effects, such as certain sounds via (one or more) speakers 24 and / or projected light or images onto (one or more) foods (e.g., via projected light or images, once the food is transferred to another plate, it appears to change color). Although some examples herein include (one or more) identifiers 22 in tableware 20, it should be appreciated that (one or more) identifiers 22 may be located in other items (including (one or more) foods). For example, a barcode may be embedded in frosting, or a QR code may be embedded in a cake topper. In addition, the identifier(s) 22 may include a specific color of the food(s), such as a sprinkle on a cookie or a red raspberry. Thus, the food(s) may be detected and / or monitored directly via the detector(s) 12 and / or the camera(s) 106, which may be coupled to the tableware 20 (e.g., cutlery) and / or in any other suitable location. This may also facilitate determining the relative position and / or interaction between the food(s) and the tableware 20 (e.g., cutlery).

[0043] In general, as noted herein, data collected by one or more sensors 30 may be provided to the controller 14. The controller 14 may then instruct the speaker(s) 24 to output sounds and / or control the display(s) 26 to project and / or display visual images and other special effects. For example, other special effects may include tactile effects generated by shaking the table and / or chair 102, causing airflow to flow at the user 104, releasing scents around the user 104, and the like. As another example, other special effects may include illuminating and / or turning off light emitters around the dining environment 100 (e.g., to darken the dining environment 100). The one or more sensors 30 may individually track the tableware 20 associated with each of the users 104 to thereby provide a personalized experience for the users 104 (e.g., when the first user 104A touches the first food on his own plate 20A, the speaker(s) 24A outputs laughter; however, when the second user 104B touches the first food on his own plate 20B, the speaker(s) 24B outputs a funny scream). The personalized experience of users 104 may be based on historical data for each of users 104, such as whether certain effects were received positively (e.g., user 104 continued to consume food) or negatively (e.g., user 104 stopped consuming food for at least a threshold period of time and / or left food on the plate at the end of the meal) during a previous dining experience. It is contemplated that any special effect disclosed herein may be provided in response to any data and / or combination of data (e.g., parameters) disclosed herein.

[0044] It should be appreciated that in addition to facilitating special effects during a dining experience, data collected by the one or more sensors 30 may also be used to inform and / or improve operations in the dining environment 100. For example, macaroni and cheese may be a popular food item ordered by users; however, data from a food weight sensor (and / or one or more of the other sensors 30, such as the camera(s) 106) may indicate that macaroni and cheese has been left on several users' plates. In response, the controller 14 may recommend and / or the restaurant may choose to change its menu and / or prepare macaroni and cheese differently to satisfy the users. In addition, the one or more sensors 30 may include a microphone(s) that captures the sounds produced by the users 104 as they consume the food(s). The controller 14 may analyze the sounds to determine whether the users 104 like the food(s) (e.g., via comparison of the sounds to sound features associated with different emotions / states). In one embodiment, the data may be input into a machine learning model that may output recommendations to adjust food recipes, menu items, etc. based on the user's eating habits and patterns over time.

[0045] Figure 4is available in Figure 1 Schematic diagram of an embodiment of a seat 102 with (one or more) speakers 24 used in the food container system 10 of FIG. As mentioned above, binaural audio can provide an immersive experience to the user by localizing the sound based on the orientation and / or movement of the user's head. In order to take into account the difference in height of the user (e.g., an adult and a child), the (one or more) speakers 24 can be automatically and / or mechanically moved based on the position and / or orientation of the user's head. For example, the seat 102 may include one or more tracks 130, and several discrete speakers 24 are arranged on the one or more tracks 130. The controller 14 may provide a signal to (one or more) actuators, and the (one or more) actuators may drive (one or more) speakers 24 to move along (one or more) corresponding tracks 130, so that (one or more) speakers 24 are appropriately positioned near the user's head to provide binaural audio. For example, if the user has a first smaller height (e.g., the user is a child), the (one or more) speakers 24 may be arranged at a lower position along (one or more) corresponding tracks 130. However, if the user has a second, greater height (e.g., the user is an adult), the speaker(s) 24 may be positioned higher along the corresponding track(s) 130. The height of the user may be gathered from historical data (e.g., which may include a user profile), input by the user and / or an operator of the dining experience, and / or determined via analysis of data obtained by one or more sensors (e.g., images captured by camera(s)). In one embodiment, the user and / or operator may manually adjust the speaker(s) 24 to position the speaker(s) near the user's head.

[0046] Considering the above, Figure 5 is used for feeding via a food system (such as Figure 1 Flowchart of one embodiment of a process 150 for generating binaural audio using a food processor system 10 of the present invention. Any suitable device (e.g., processor(s) 16) may perform process 150. In one embodiment, process 150 may be implemented by executing instructions stored in a non-transitory computer-readable medium (e.g., memory device(s) 18). Although process 150 is described using blocks in a particular order, it should be understood that the present disclosure contemplates that the blocks may be performed in any suitable order, that certain blocks may be omitted, and / or that other blocks may be added.

[0047] At block 152, the processor(s) may identify an interaction between a food item and a piece of cutlery (e.g., a piece of cutlery-type cutlery, such as a fork, knife, or spoon). For example, the processor(s) may receive data from one or more sensors, including data indicating an identifier coupled to the piece of cutlery, data indicating a location of the piece of cutlery, data indicating movement of the piece of cutlery, and / or data indicating a type of food item, etc. The interaction may include contact between the food item and the piece of cutlery, initiating and / or maintaining (e.g., for a threshold time, such as one second) movement of the piece of cutlery toward the food item, reaching a threshold distance between the food item and the piece of cutlery, and / or conveying the food item toward a user's mouth via the piece of cutlery, etc. For example, data from one or more sensors (e.g., signals from the detector(s) and / or images captured by a camera) may indicate that a user has inserted a spoon into a soup bowl to contact soup, that a user is moving a knife toward a piece of bread, and / or that the knife is within a threshold distance of the piece of bread, and / or that a user is conveying a bite of cake toward the user's mouth.

[0048] At box 154, the processor(s) may instruct the speaker(s) to provide an audible output in response to the interaction. In one embodiment, the processor(s) may instruct the speaker(s) to provide binaural audio so that the audio data appears to be projected from the food. For example, when the user contacts the cake, the speaker(s) may output sound to provide the effect of the cake singing "Happy Birthday". The processor(s) may also control one or more actuators to move the speaker(s), as described herein. The speaker(s) may be located on a seat occupied by a user and / or in any other suitable location around the dining environment.

[0049] At box 156, the processor(s) may instruct the one or more display devices to provide a visible output in response to the interaction. In one embodiment, the processor(s) may instruct the display(s) to project an image onto and / or near the food (e.g., onto a plate supporting the food; onto a cup holding the food; onto a table supporting a plate or cup with the food; onto a user or another user at the table). For example, when the user touches the cake, the display(s) may project an image of a monster onto the cake. The processor(s) may also coordinate sounds from the speaker(s) with images from the display(s). For example, the sounds may include phrases and / or laughter, and the image may include a monster whose mouth moves substantially synchronously with the phrases and / or laughter. The display(s) may be located on the ceiling and / or any other suitable location in the dining environment.

[0050] At box 158, the processor(s) may instruct the one or more haptic devices to provide a tactile output in response to the interaction. For example, the processor(s) may instruct the haptic device(s) to shake a table supporting food, cause airflow to flow at the user, release an odor near the user, etc. The processor(s) may also coordinate sounds from the speaker(s), images from the display(s) and / or tactile outputs from the haptic device(s). For example, the image may include a flame, and the tactile output may include a stream of warm air that rushes toward the user substantially synchronously with the flame. The haptic device(s) may be located at a table, a chair, tableware, a ceiling, a wall, and / or any other suitable location in a dining environment. In fact, in one embodiment, one or more sensors, speakers, display(s) and / or haptic devices may be coupled to a table, a chair, a ceiling, a wall, tableware (e.g., cutlery type tableware, a plate, a cup), and / or any other suitable location in a dining environment.

[0051] It should be appreciated that the food utensil system can provide any of a variety of interactive special effects in response to any of a variety of events in a dining environment. For example, (one or more) processors can track the movement of the dish before it is presented to the user and / or when the dish is delivered to the user. Based on determining that the dish has been delivered to the user (e.g., set on the table), (one or more) processors can instruct (one or more) speakers and / or (one or more) tactile devices to generate auditory and tactile stimulations, respectively. For example, in response to the dish being set on the table and / or when the dish is set on the table, the user can hear a loud sound effect and feel his seat shaking. In one embodiment, (one or more) processors can create a storyline during the dining experience using binaural audio, visual images and / or tactile stimulation. Different meals (e.g., appetizers, main dishes, desserts) of the dining experience can correspond to different parts of the storyline. Other variations are also envisioned. For example, the food utensil system may also be configured to more generally identify the interaction between the food and the user via the user's hand (e.g., the user picks up a cookie), via other movements (e.g., chewing movements), and / or via the piece of tableware (e.g., the user moves a cup to drink a drink, the user moves a knife to cut bread). In response, the food utensil system may provide any of a variety of special effects, such as any of the special effect(s) disclosed herein.

[0052] Although only certain features of the present disclosure have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the present disclosure.

[0053] The technology presented and claimed herein is cited and applied to substantial objects and specific examples that demonstrably improve the practical nature of the art, and as such is not abstract, intangible, or purely theoretical. In addition, if any claim appended to the end of this specification contains one or more elements designated as "a means for [performing] ... [function]" or "a step for [performing] ... [function]," it is intended that such elements are to be interpreted in accordance with 35 U.S.C. 112(f). However, for any claim containing elements designated in any other manner, it is intended that such elements are not to be interpreted in accordance with 35 U.S.C. 112(f).

Claims

1. A food container system, comprising: an identifier coupled to a utensil configured to be used by a user to engage a food product; a detector configured to generate data indicative of a position of the appliance, movement of the appliance, or both based on detection of the identifier; as well as A controller is communicatively coupled to the detector and is configured to instruct a speaker to generate audio data based on the position of the instrument, the movement of the instrument, or both.

2. The food container system according to claim 1, wherein: The audio data includes binaural audio data to provide an effect that the audio data appears to be projected from the food.

3. The food container system according to claim 1, wherein: The identifier includes a radio frequency identification (RFID) tag, a near field communication (NFC) tag, a barcode, a quick response (QR) code, or any combination thereof.

4. The food container system according to claim 1, wherein: The detector includes a radio frequency identification (RFID) reader, a near field communication (NFC) reader, a barcode scanner, a quick response (QR) code scanner, or any combination thereof.

5. The food utensil system of claim 1 , comprising a camera configured to capture images representing the position of the utensil, the movement of the utensil, user movement of the user, the type of food, the food position of the food, the food movement of the food, or any combination thereof.

6. The food container system according to claim 1, comprising: a food displacement sensor configured to detect food movement of the food product; a weight sensor configured to detect the weight of the food; a motion sensor configured to detect user movement of the user, the movement of the appliance, or both; or Any combination thereof.

7. The food container system of claim 1, comprising the speaker coupled to a chair configured to support the user, wherein: The controller is configured to cause the actuator to move the position of the speaker relative to the user's head.

8. The food container system according to claim 1, wherein: The controller is configured to dynamically project an image onto the food product, onto tableware supporting the food product, or both based on the position of the utensil, the movement of the utensil, or both.

9. The food container system according to claim 1, wherein: The controller is configured to instruct a haptic device to provide tactile stimulation based on the position of the implement, the movement of the implement, or both.

10. The food container system according to claim 1, wherein: The controller is configured to instruct the speaker to generate the audio data in response to the position of the utensil, the movement of the utensil, or both that indicates contact between the utensil and the food product.

11. The food container system of claim 1, wherein: The detector is coupled to a pan supporting the food product.

12. A method for providing an immersive dining experience, the method comprising: identifying, via the one or more processors and based on data obtained by the one or more sensors, an interaction between the food product and the user; as well as In response to the interaction between the food item and the user, an image is projected onto the food item via a projector.

13. The method of claim 12, comprising outputting audio data via a speaker in response to the interaction. The method of claim 12 , comprising outputting a tactile stimulus via a haptic device in response to the interaction.

15. The method of claim 12, wherein: Identifying the interaction between the food product and the user includes identifying contact between the food product and an implement.

16. The method of claim 12, wherein: Identifying the interaction between the food and the user includes: identifying the initiation of movement of an utensil held by the user toward the food, maintained movement of the utensil toward the food for a threshold time, the utensil being within a threshold distance of the food and / or the food moving toward the user's mouth via the utensil, or any combination thereof.

17. The method of claim 12, wherein: The data includes images captured by a camera, proximity data obtained by one or more proximity sensors, relative position data obtained by one or more radio frequency identification (RFID) readers, or any combination thereof.

18. A food container system, comprising: one or more sensors configured to obtain data indicative of an interaction between the food product and a user; as well as A controller is communicatively coupled to the one or more sensors and is configured to instruct a projector to project an image onto the food product in response to the interaction between the food product and the user.

19. The food container system of claim 18, wherein: The controller is configured to instruct a speaker to generate audio data in response to the interaction between the food item and the user.

20. The food container system of claim 18, wherein: The controller is configured to select the image based on a type of the food product, historical data associated with the user, or both.

Images