Olfactory tags for chat and augmented reality-based messaging

Abstract

An olfactory tag for chats between electronic devices to make messaging more immersive, personalized, and realistic by combining olfactory information with traditional text-based formats and augmented reality (AR) messaging. The olfactory label is used for indicating that the message comprises olfactory information to a receiver. The olfactory tag shows a graphical representation of a specific odor, which may be sent and received by chat or AR messages. The olfactory tag provides recipients with control to acquire the transmitted odour at a desired time. This is particularly useful because certain olfactory information (i.e., odor) may be very direct and invasive. The olfactory tag is activated (i.e., releases its odor) when tapped or rubbed by a recipient.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. application serial number 17 / 868,395, filed on July 19, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This topic relates to interaction between electronic devices, such as smart devices. Background Art

[0004] Olfactory families allow for the classification of fragrances according to their key olfactory characteristics. Olfactory families are created by grouping raw materials together, such as flowers, woods, aromatics or citrus fruits, or by drawing inspiration from traditional accords, such as oriental or chypre.

[0005] BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings depict one or more implementations by way of example only, and not by way of limitation.In the drawings, like reference numerals represent like or similar elements.

[0007] The features of the various implementations disclosed will be readily understood from the detailed description below with reference to the accompanying drawings. In the specification and in the several views of the drawings, each element is represented by a single reference numeral. When multiple similar elements are present, a single reference numeral may be assigned to the similar element, with the additional letter representing the specific element.

[0008] Unless otherwise noted, the various elements shown in the figures are not drawn to scale. For clarity, the dimensions of various elements may be exaggerated or reduced. Several figures depict one or more implementations and are presented by way of example only and should not be construed as limiting. The drawings include the following figures:

[0009] Figure 1 is a high-level functional block diagram of an example smartphone configured to receive and transmit olfactory information;

[0010] Figure 2 It is the user interface on the smartphone that displays the olfactory tags;

[0011] Figure 3 is an augmented reality (AR) user interface that displays olfactory tags; and

[0012] Figure 4 is a flow chart of a method for sending and receiving olfactory tags. DETAILED DESCRIPTION

[0013] The transmission of olfactory information makes messaging between smart electronic devices more immersive, personalized, and realistic by combining olfactory information with traditional text-based formats and AR messaging. Olfactory tags are used to indicate to the recipient that olfactory information is included in the message. Olfactory tags show a graphical representation of a specific smell, which can be sent and received via chat or AR messaging. Olfactory tags provide the recipient with control over obtaining the transmitted smell at the desired time. This is particularly useful because some olfactory information (i.e., smells) can be very direct and intrusive. The olfactory tag is activated (i.e., releases its scent) when it is touched by the recipient (such as by tapping or rubbing).

[0014] As used herein, the terms "connect," "connected," "couple," and "coupled" refer to any logical, optical, physical, or electrical connection, including a link, through which an electrical or magnetic signal generated or provided by one system element is transmitted to another coupled or connected system element. Unless otherwise specified, coupled or connected elements or devices are not necessarily directly connected to each other and may be separated by intermediate components, elements, or communication media, one or more of which can modify, manipulate, or carry electrical signals. The term "on" means directly supported by an element or indirectly supported by the element through another element that is integrated into, or supported by, the element.

[0015] Other objects, advantages and novel features of the examples will be set forth in part in the following description and in part will become apparent to those skilled in the art after consideration of the following and accompanying drawings, or may be learned by production or operation of the examples. The objects and advantages of the present subject matter may be realized and obtained by the methods, means and combinations particularly pointed out herein.

[0016] Reference will now be made in detail to the examples illustrated in the accompanying drawings and discussed below.

[0017] Figure 1 A high-level functional block diagram of an example mobile device in a sample configuration is shown. As shown, smartphone 100 includes flash memory 110 that stores programming to be executed by CPU 120 to perform all or a subset of the functions described herein. Figure 1 As shown, the CPU 120 of the smartphone 100 includes a mobile display driver 130, a user input layer 140 (e.g., a touch screen) for a front-facing image display 145, a display controller 150, a front-facing visible light camera 155, and one or more rear-facing visible light cameras 160 with substantially overlapping fields of view. In such a configuration, the flash memory 110 may further include a plurality of images or videos generated via the cameras.

[0018] like Figure 1 As shown, the smartphone 100 may further include at least one digital transceiver (XCVR) 165, shown as a wireless wide area network transceiver (Wireless Wide Area Network XCVRs, WWAN XCVRs), for digital wireless communication via a wide area wireless mobile communication network. The smartphone 100 may also include additional digital or analog transceivers, such as for short-range network communication (such as via Near Field Communication (NFC), Visible Light Communication (VLC), Digital Enhanced Cordless Telecommunication (DECT), ZigBee, or For example, the short-range XCVRs 170 may take the form of any available two-way wireless local area network (WLAN) transceiver of a type compatible with one or more standard communication protocols implemented in wireless local area networks, such as IEEE 802.11. In some configurations, XCVRs 170 may also be configured to communicate with a global event database.

[0019] In order to generate location coordinates for positioning of the smartphone 100, the smartphone 100 may also include a global positioning system (GPS) receiver. Alternatively or additionally, the smartphone 100 may utilize either or both of the short-range XCVRs 170 and the WWAN XCVRs 165 to generate location coordinates for positioning. For example, based on a cellular network, or Positioning systems can generate very precise location coordinates, especially when used in combination.

[0020] The transceivers 165 and 170 (i.e., network communication interfaces) may conform to one or more of the various digital wireless communication standards used by modern mobile networks. Examples of WWAN transceivers 165 include, but are not limited to, transceivers configured to operate in accordance with Code Division Multiple Access (CDMA) and 3rd Generation Partnership Project (3GPP) network technologies, including, for example, but not limited to, 3GPP Type 2 (or 3GPP2) and Long-Term Evolution (LTE, sometimes referred to as "4G"), or 5G New Radio (referred to as "5G"). For example, the transceivers 165 and 170 provide two-way wireless communication of information, including digitized audio signals, still images and video signals, web page information for display and network-related input, and various types of mobile messaging communications to and from the smartphone 100.

[0021] The smartphone 100 also includes a microprocessor serving as a central processing unit (CPU), such as Figure 1 120 . A microprocessor is a circuit having elements that are constructed and arranged to perform one or more processing functions, typically various data processing functions. Although discrete logic components can be used, these examples utilize components that form a programmable CPU. For example, a microprocessor includes one or more integrated circuit (IC) chips that include electronic components that perform the functions of the CPU 120. For example, the CPU 120 can be based on any known or available microprocessor architecture, such as Reduced Instruction Set Computing (RISC) using an ARM architecture, as commonly used today in mobile devices and other portable electronic devices. Of course, other arrangements of microprocessor circuits can be used to form the CPU 120 or microprocessor hardware in smart watches, smartphones, laptop computers, and tablet computers.

[0022] The CPU 120 acts as a programmable host controller for the smartphone 100 by configuring the smartphone 100 to perform various operations according to instructions or programming executable by the CPU 120. For example, such operations may include various general operations of the smartphone 100, as well as operations related to application programming on the smartphone 100. Although microprocessors can be configured through the use of hard-wired logic, typical microprocessors in mobile devices are general-purpose processing circuits that are configured by executing programming.

[0023] Smartphone 100 also includes a memory or storage system for storing programming and data. Figure 1 In the example shown, the memory system may include flash memory 110, random-access memory (RAM) 180, a local event database, and other memory components as needed. RAM 180 may be used as short-term storage for instructions and data processed by CPU 120, such as working data processing memory, while flash memory 110 generally provides long-term storage.

[0024] Thus, in the example of smartphone 100, flash memory 110 is used to store programming or instructions that are executed by CPU 120. Depending on the type of device, smartphone 100 stores and runs a mobile operating system, which executes specific applications. Examples of mobile operating systems include Google Android, Apple iOS (used in iPhone or iPad devices), Windows Mobile, Amazon Fire OS, RIM BlackBerry OS, etc.

[0025] In a sample configuration, CPU 120 may construct a map of the environment surrounding smartphone 100, determine the location of smartphone 100 within the mapped environment, and determine the relative position of smartphone 100 to one or more objects in the mapped environment. CPU 120 may construct the map and determine location and position information using a simultaneous localization and mapping (SLAM) algorithm applied to data received from one or more sensors.

[0026] The sensor data may include images received from cameras 155 and 160, distances received from a laser rangefinder, position information received from a GPS unit, motion and acceleration data received from an inertial measurement unit (IMU) 190, or a combination of data from these sensors or from other sensors that provide data for determining position information. In the context of augmented reality, SLAM algorithms are used to build and update a map of the environment while tracking and updating the position of the device (or user) in the mapped environment. The mathematical solution can be approximated using different statistical methods such as particle filtering, Kalman filtering, extended Kalman filtering, and covariance intersection. In a system including a high-definition (HD) camera that captures video at a high frame rate (e.g., thirty frames per second), the SLAM algorithm updates the map and the position of objects at least as frequently as the frame rate, in other words, calculates and updates the map and positioning thirty times per second.

[0027] The smartphone 100 is configured to send and receive messages to and from a remote smartphone device via short-range XCVRs 170 and WWAN XCVRs 165. The messaging supports traditional chat-based messaging, augmented reality (AR) messaging, and hybrid chat-AR messaging. The smartphone 100 also includes an olfactory transducer 200 configured to store and disperse small amounts of one or more stored scents for perception by a user near the olfactory transducer 200. The CPU 120 is configured to send control signals to the olfactory transducer 200 to control the dispersion of the scents. In one example, the olfactory transducer 200 is built into the smartphone 100. In other examples, a separate device containing the olfactory transducer 200 is coupled to the smartphone 100, such as a box coupled to the smartphone 100.

[0028] Figure 2A user interface 202 is shown forming the user input layer 140 on the display 145 of the smartphone 100 and displaying an olfactory tag 204. The user interface 202 displays messages 210 (such as chats) sent by the user of the smartphone 100 to the remote smartphone via the short-range XCVRs 170 and the WWAN XCVRs 165, as well as messages 212 received by the smartphone 100 and viewable by the user. The user interface 202 is configured to send and receive olfactory tags 204 encoded with olfactory information corresponding to a particular odor. The encoded olfactory information is configured to activate the olfactory transducer 200 to release the corresponding odor when the user engages the olfactory tag 204. The olfactory tag 204 graphically presents the encoded olfactory information to the user of the smartphone 100. The olfactory tag 204 includes a visual preview 206 of the transmitted odor. In an example, the visual preview 206 can be an icon, a symbol, a cartoon, a picture, an emoticon, Text or similar visual information. Figure 2 In the example shown, the visual preview 206 is a cartoon pineapple that is used to represent the smell of the pineapple sent to the smartphone 100. A variety of images can be used as the visual preview 206 of the olfactory tag 204. For example, a vanilla flower can be used to represent the smell of vanilla, a chocolate cake can be used to represent the smell of chocolate, and a strawberry can be used to represent the smell of strawberry.

[0029] The olfactory tag 204 may also include textual instructions 208 that instruct how to engage and activate the olfactory tag 204 to release a scent through the olfactory transducer 200. Figure 2 In the example shown, text instruction 208 reads "Rub me!" to indicate to the user that the olfactory tag 204 displayed on user interface 202 must be rubbed to release the scent represented by olfactory tag 204. Other text, such as "Tap me!", may be used to indicate to the user that the olfactory tag 204 must be tapped to release the scent represented by olfactory tag 204. User interface 202 is configured to detect interactions between the user and olfactory tag 204. The intensity of tapping or rubbing the displayed olfactory tag 204 adjusts the amount of scent released by olfactory transducer 200 so that the user is not overwhelmed by the received scent. For example, a single tap of the displayed olfactory tag 204 results in a small, predetermined amount of scent being released from olfactory transducer 200, while two taps result in twice the amount of scent being released compared to a single tap. In the example of rubbing the olfactory tag 204, rubbing for two seconds results in a small, predetermined amount of scent being released, while rubbing for five seconds results in twice the amount of scent being released compared to rubbing for two seconds. The configuration of the friction olfactory tag 204 emulates similar physical scent-based tags that are activated by rubbing or scratching to provide an intuitive user interface.

[0030] Figure 3A sender smartphone 100A is shown with a display 145A including a user interface 202A, and a receiver smartphone 100B with a display 145B including a user interface 202B. User interface 202A and receiver user interface 202B are configured to display messaging (such as AR chat) between the smartphones. The sender smartphone 100A is configured to display the AR chat on user interface 202A based on an image generated by the rear camera 160 of the sender smartphone 100A. As shown, the sender selects an object 310 (a pineapple) within the field of view of the rear camera 160 to create a clone image 312 (i.e., a "segmented image") of the object 310. The sender smartphone 100A includes an odor sensor or image classification functionality to determine whether information indicating an odor can also be sent with the message. In the example of sending the pineapple image 312, information indicating the pineapple's smell is sent along with the message in the form of an olfactory tag 204. The sender smartphone 100A sends the clone image 312 and the olfactory tag 204 to the receiver smartphone 100B. The receiving smartphone 100B displays the cloned image 312 with the olfactory tag 204 on the user interface 202B of the receiving smartphone 100B. In this example, the displayed olfactory tag 204 partially overlaps the displayed cloned image 312. In other examples, the olfactory tag 204 may be displayed to completely overlap the cloned image 312, or separately from the cloned image 312. In another example, the sender may choose to send only the olfactory tag 204 without the cloned image. In this example, two users are chatting in close proximity, and the two users can use a shared, physically separate olfactory transducer 200 to disseminate the received scent.

[0031] When the olfactory tag 204 is engaged by the user of the receiving smartphone 100B, the olfactory tag 204 instructs the olfactory transducer 200 to emit a corresponding odor, such as that previously described with reference to FIG. Figure 2 As depicted, the olfactory tag 204 is revealed by tapping or rubbing.

[0032] In one example, the user of a sending smartphone 100A can select which olfactory tag 204 to send from a displayed set of predefined olfactory tags 204, which are associated with predefined scents stored in the memory 110 of the sending smartphone 100A. The receiving smartphone 100B can access the same or a different set of predefined olfactory tags 204 to easily preview what specific scent was received without activating the olfactory tags 204. For example, an image of a pine tree corresponds to the fresh smell of pine trees for both the smartphone user and the sender. The sending smartphone 100A sends an olfactory tag 204 representing a pine tree to the receiving smartphone 100B, and the receiving smartphone 100B, without activating the olfactory tag 204, learns from the predefined olfactory tags 204 that an olfactory tag 204 representing the fresh smell of pine trees has been sent. In another example, the user of the sending smartphone 100A can select a predefined scent to accompany a customized olfactory tag. For example, the user selects an image of a candle as the visual preview 206 of the olfactory tag 204 and selects citrus as the scent of the olfactory tag 204. The receiving smartphone 100B receives the customized olfactory tag 204 and does not know the scent of the olfactory tag 204 until the olfactory tag 204 is activated and released by the olfactory transducer 200.

[0033] Figure 4 is a flow chart 400 illustrating a method of sending and receiving messages including an olfactory tag 204 between two smartphones 100 .

[0034] At block 402, a user of a sending smartphone 100A creates and sends a chat message with an olfactory tag 204 to a receiving smartphone 100B. The processor 120 of the sending smartphone 100A sends the message via short-range XCVRs 170 or WWAN XCVRs 165. The message sent can be a traditional chat-based message or an AR message. In one example, the sending smartphone 100A sends an olfactory tag 204 selected from a set of predefined olfactory tags 204 with predefined scents. In another example, the sender selects the scent that accompanies the olfactory tag 204.

[0035] At block 404, the recipient smartphone 100B receives the message forwarded from the sender smartphone 100A via the corresponding short-range XCVRs 170 or WWAN XCVRs 165. The message contains the olfactory tag 204 with the encoded olfactory information. The recipient smartphone 100B displays the olfactory tag 204 on the display 145B via the recipient smartphone 100B user interface 202B.

[0036] At block 406, the user of the recipient smartphone 100B activates the olfactory tag 204 by interacting with the displayed olfactory tag 204. For example, the user taps or rubs the displayed olfactory tag 204. The user can control the intensity of the interaction with the displayed olfactory tag 204 to determine the intensity of the released scent.

[0037] At block 408, the processor 120 of the recipient smartphone 100B sends a signal to the olfactory transducer 200 to release an odor corresponding to the encoded olfactory information in the olfactory tag 204 and the intensity of the user's interaction with the olfactory tag 204. The user of the recipient smartphone 100B is then able to perceive the released odor.

[0038] It should be understood that the terms and expressions used herein have the ordinary meaning consistent with those terms and expressions in their respective fields of investigation and study, unless otherwise specifically stated herein. Relational terms such as first and second may be used only to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between those entities or actions. The terms "comprises," "comprising," "includes," "including," or any other variant thereof are intended to encompass non-exclusive inclusions such that a process, method, article, or apparatus comprising or containing a series of elements or steps includes not only those elements or steps, but may also include other elements or steps that are not explicitly listed or that are inherent to such a process, method, article, or apparatus. Without further limitation, an element preceded by "a" or "an" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0039] Unless otherwise indicated, all measurements, values, levels, positions, quantities, dimensions, and other specifications listed in this specification (including the appended claims) are approximate and not exact. These quantities are intended to have a reasonable range that is consistent with the functions to which they are related and the customary practices in the art to which they belong. For example, unless expressly stated, parameter values, etc. may vary from the stated quantity by as much as ±10%.

[0040] Furthermore, in the foregoing detailed description, various features are grouped together in various examples for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed examples require more features than expressly recited in each claim. Rather, as reflected in the following claims, there is less than all features of any single disclosed example to be claimed. Accordingly, the following claims are hereby incorporated into the detailed description, with each claim standing on its own as separately claimed subject matter.

[0041] While what is considered to be the best mode and other examples have been described above, it should be understood that various modifications may be made therein, and the subject matter disclosed herein may be implemented in a variety of forms and examples, and may be used in many applications, only some of which are described herein. The following claims are intended to claim any and all modifications and variations that fall within the true scope of the present concept.

Claims

1. An electronic device, comprising: processor; a transceiver coupled to the processor and configured to receive a message from a remote electronic device, the message including an image representing the scent and information; a display configured to display the image as an olfactory tag; as well as An olfactory transducer is coupled to the processor and is configured to release the scent when a user interacts with the olfactory tag.

2. The electronic device according to claim 1, wherein: The olfactory tag is indicative of the odor.

3. The electronic device according to claim 1, wherein: The display includes a touch screen operable as a user interface.

4. The electronic device according to claim 3, wherein: The touch screen is configured to respond to a user tapping an olfactory tag displayed on the touch screen.

5. The electronic device according to claim 3, wherein: The touch screen is configured to respond to a user rubbing an olfactory tag displayed on the touch screen.

6. The electronic device according to claim 3, wherein: The intensity of the scent corresponds to the intensity of the user's interaction with the touch screen.

7. The electronic device according to claim 1, wherein: The olfactory tag is configured to be displayed in augmented reality.

8. The electronic device according to claim 7, wherein: The message is configured to be sent in a shared augmented reality session with the remote electronic device.

9. The electronic device according to claim 1, wherein: The image is configured to be selected from a predefined set of images.

10. A method for releasing an odor using an electronic device, the electronic device comprising a processor, a transceiver, a display, and an olfactory transducer, the transceiver being coupled to the processor and configured to receive a message from a remote electronic device, the message comprising an image and information representing an odor, the display being configured to display the image as an olfactory tag, the olfactory transducer being coupled to the processor, the method comprising: receiving, by the processor, the image representing the odor and the information; displaying, by the processor, the image as the olfactory tag on the display; enabling, by the processor, a user to interact with the olfactory tag; as well as In response to the user interacting with the olfactory tag, the scent is released from the olfactory transducer.

11. The method according to claim 10, wherein: The olfactory tag is indicative of the odor.

12. The method according to claim 10, wherein: The display includes a touch screen operable as a user interface.

13. The method according to claim 12, wherein: The touch screen responds to the user tapping an olfactory tag displayed on the touch screen.

14. The method according to claim 12, wherein: The touch screen responds to the user rubbing an olfactory tag displayed on the touch screen.

15. The method according to claim 12, wherein: The intensity of the scent corresponds to the intensity of the user's interaction with the touch screen.

16. The method according to claim 10, wherein: The olfactory tag is displayed in augmented reality.

17. The method according to claim 16, wherein: The message is sent in a shared augmented reality session with the remote electronic device.

18. The method according to claim 10, wherein: The image is selected from a predefined set of images.

19. A non-transitory computer readable medium storing program code, the program code being operable, when executed, to cause an electronic processor of an electronic device to perform operations, wherein the electronic device comprises a transceiver, a display, and an olfactory transducer, the transceiver being coupled to the processor and configured to receive a message from a remote electronic device, the message comprising an image representing an odor and information, the display being configured to display the image as an olfactory tag, the olfactory transducer being coupled to the processor, the operations being: receiving said image representing said odor and said information; displaying the image as the olfactory tag on the display; detecting a user interacting with the olfactory tag; as well as In response to detecting the user's interaction with the olfactory tag, a control signal is sent to the olfactory transducer to release the scent.

20. The non-transitory computer readable medium storing program code of claim 19, wherein the image is indicative of the smell.

Images