METHOD AND APPARATUS FOR OUTPUTTING VIRTURAL SMOKE IMAGE / VIDEO

Abstract

In order to output a virtual smoke image, the electronic device receives a connection request from the aerosol generating device while being paired with one or more external controllers, determines a controller type of the aerosol generating device, and if the determined controller type corresponds to the type of a target external controller among the one or more external controllers, interrupts pairing with the target external controller, connects to the aerosol generating device, receives detection information regarding the status of the aerosol generating device, generates a virtual smoke image based on the detection information, and outputs the virtual smoke image via a display of the electronic device.

Description

[Technical field]

[0001] The following embodiments relate to techniques for implementing virtual smoke in electronic devices. [Background technology]

[0002] In recent years, virtual reality, augmented reality, and mixed reality technologies that apply computer graphics technology have been developed. In this regard, virtual reality technology refers to a technology that uses a computer to create a virtual space that does not exist in the real world and makes the virtual space feel real, while augmented reality or mixed reality technology refers to a technology that adds and expresses information generated by a computer in the real world, that is, a technology that allows interaction with a user in real time by combining the real world and the virtual world.

[0003] Augmented reality and mixed reality technologies are being used in combination with technologies in various fields. In the field of electronic cigarettes, there is an increasing demand for devices that realize virtual reality or augmented reality services, and various research projects are being conducted in response to this demand. Summary of the Invention [Problem to be solved by the invention]

[0004] One embodiment may provide a method for outputting a virtual smoke image performed by an electronic device.

[0005] One embodiment may provide an electronic device for performing the method of outputting a virtual smoke image. [Means for solving the problem]

[0006] A virtual smoke image output method performed by an electronic device according to one embodiment may include an operation of outputting a virtual image using a display of the electronic device so that an avatar corresponding to a user of the electronic device appears in a virtual space, an operation of receiving detection information regarding a state of an aerosol generating device connected to the electronic device, an operation of determining a reference movement of the user based on the detection information and controlling a target movement of the avatar so that the determined reference movement appears, an operation of generating virtual smoke information based on smoking information of the detection information, and an operation of outputting the virtual image so that virtual smoke appears in the virtual image based on the virtual smoke information.

[0007] The avatar's face may be generated based on a graphic image of the user's face, and the avatar's facial expressions may change based on facial expression information generated by an additional sensor.

[0008] The operation of generating virtual smoke information based on the smoking information of the detection information may include an operation of determining attributes of virtual smoke based on the smoking information of the detection information (the attributes of the virtual smoke including at least one of a size, a shape, and a duration of the virtual smoke), and an operation of generating the virtual smoke information based on the attributes of the virtual smoke.

[0009] The operation of outputting the virtual image so that virtual smoke appears may include an operation of determining an exhalation point at which a user of the aerosol generating device begins to exhale based on the smoking information of the detection information, and an operation of outputting the virtual image so that the virtual smoke appears at the exhalation point.

[0010] The operation of determining an exhalation time at which a user of the aerosol generating device begins to exhale based on the smoking information of the detection information may include an operation of determining the exhalation time based on a first time at which a distance between one side of the aerosol generating device and a part of the user's body reaches within a first threshold, sensed by the aerosol generating device, and a second time after the first time at which the distance between the one side of the aerosol generating device and a part of the user's body exceeds a second threshold.

[0011] The operation of determining the exhalation time at which a user of the aerosol generating device begins to exhale based on the smoking information of the detection information may include an operation of determining the exhalation time based on a change in capacitance of the aerosol generating device.

[0012] The operation of determining the exhalation time based on a change in the capacitance of the aerosol generating device may include an operation of determining the exhalation time based on a first time at which a value of the capacitance of the aerosol generating device sensed by the aerosol generating device reaches within a first threshold and a second time after the first time at which the value of the capacitance of the aerosol generating device exceeds a second threshold.

[0013] The detection information can be received from the aerosol generating device that has established a wireless communication channel with a dongle connected to the electronic device.

[0014] The detection information can be received from the aerosol generating device which has established a wireless communication channel directly with the electronic device.

[0015] The operation of outputting a virtual image so that the avatar appears includes: The method may include receiving information about a second avatar of a second user from a server connected to the electronic device, and outputting a virtual image such that the second avatar appears in the virtual space.

[0016] An electronic device performing a method for outputting a virtual smoke image according to one embodiment includes a display, a memory having a program recorded thereon for outputting a virtual smoke image, and a processor for executing the program, and the processor can perform the following operations in the method for outputting a virtual smoke image performed by the electronic device: outputting a virtual image using the display of the electronic device so that an avatar corresponding to a user of the electronic device appears in a virtual space; receiving detection information regarding a state of an aerosol generating device connected to the electronic device; determining a reference movement of the user based on the detection information and controlling a target movement of the avatar so that the determined reference movement appears; generating virtual smoke information based on smoking information of the detection information; and outputting the virtual image so that virtual smoke appears in the virtual image based on the virtual smoke information. Effect of the Invention

[0017] A method may be provided for outputting a virtual smoke image via a display of an electronic device.

[0018] A method for using an aerosol generating device as a controller for an electronic device may be provided. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram for explaining a virtual smoke image output system according to an embodiment. [Diagram 2] 1 is an exemplary diagram of a display output screen of an electronic device in accordance with an embodiment. [Diagram 3] 1A to 1C are diagrams illustrating a user's smoking behavior divided into sections and a method for predicting a user's exhalation time and outputting a virtual smoke image according to an embodiment of the present invention. [Figure 4] 1 is a flowchart of a method for outputting a virtual smoke image according to an embodiment. [Diagram 5] FIG. 1 illustrates an example of an aerosol generating device according to one embodiment. [Figure 6]1 is a flowchart of a method for generating virtual smoke information according to one embodiment. [Figure 7] 1 is a flowchart of a method for outputting a virtual smoke image at the time of exhalation according to an embodiment. [Figure 8] 1 is a flowchart of a method for outputting a virtual smoke image so that a second avatar appears in a virtual space according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The specific structural or functional description of the embodiment is disclosed for the purpose of illustration only, and can be modified and implemented in various forms. Therefore, the actual implementation is not limited to the specific embodiment disclosed, and the scope of the present specification includes modifications, equivalents, or alternatives included in the technical ideas described in the embodiment.

[0021] Although terms such as first or second may be used to describe various components, these terms should be construed only for the purpose of distinguishing one component from another component, for example, a first component may be named a second component, and similarly, a second component may also be named a first component.

[0022] When a component is referred to as being "coupled" to another component, it may be directly coupled or connected to the other component, but it should be understood that there may be other components in between.

[0023] The singular term includes the plural term unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to specify the presence of a stated feature, number, step, operation, component, part, or combination thereof, and should not be understood to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0024] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted as meanings consistent with the meanings they have in the context of the relevant art, and should not be interpreted as ideal or overly formal meanings unless expressly defined in this specification.

[0025] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are used to designate the same components regardless of the drawing numerals, and duplicated description thereof will be omitted.

[0026] FIG. 1 is a diagram for explaining a virtual smoke image output system according to an embodiment.

[0027] 1, a system 100 for outputting a virtual smoke image can include an electronic device 110 for outputting a virtual smoke image, an aerosol generating device 120, and a dongle 130. A user 10 using the system 100 can wear the electronic device 110 and smoke via the aerosol generating device 120. "Smoking" can refer to heating, burning, and the like that causes the emission of smoke or aerosol from a smokable material.

[0028] In one embodiment, the electronic device 110 may provide a user with a virtual reality, augmented reality, or metaverse image in which tobacco smoke is emitted.

[0029] In an embodiment, the electronic device 110 may output a virtual image in which an avatar corresponding to the user 10 appears. The electronic device 110 according to an embodiment may realize virtual smoke through an image in conjunction with a smoking behavior of the user 10 via the aerosol generating device 120. For example, a virtual image in which an avatar corresponding to the user 10 smokes in a virtual space in which virtual smoke appears may be output by the electronic device 110.

[0030] In one embodiment, the aerosol generating device 120 can be named an electronic cigarette device, an inhaler, or a smoking stick. The structure and specific operation method of the aerosol generating device 120 will be described in detail with reference to FIG. 5 below.

[0031] In one embodiment, a user may use the aerosol generating device 120 to engage in the act of smoking, which may result in smoke 142 being generated by the breath of the user 10. In one embodiment, the aerosol generating device 120 may provide an aerosol to the user 10 by heating an aerosol product within the aerosol generating device 120 in a variety of ways. For example, the aerosol product may be a stick that is inserted into the aerosol generating device 120. As another example, the aerosol product may be a replaceable cartridge that is inserted into the aerosol generating device 120.

[0032] According to an embodiment, visually observable smoke 142 may be generated by the breath of the user 10 who inhales the aerosol provided to the aerosol generating device 120, but is not limited thereto. For example, smoke 142 may not be generated by the breath of the user 10.

[0033] According to an example, when a user smokes using the aerosol generating device 120 but smoke 142 is not generated, virtual smoke may be outputted to appear in the virtual image. According to an example, even when a user does not actually smoke using the aerosol generating device 120, virtual smoke may be outputted to appear in the virtual image. For example, the virtual smoking program of the electronic device 110 may provide a specific mode that can generate virtual smoke information based on smoking information of the detection information received from the aerosol generating device 120 and output virtual smoke to appear in the virtual image. As a result, even when smoke 142 is not visually observed in reality from the exhaled breath of the user 10 who smokes using the aerosol generating device 120, the user 10 can observe the virtual image in which virtual smoke appears outputted by the electronic device 110 through virtual reality or augmented reality. However, the present disclosure is not limited thereto.

[0034] According to an embodiment, the electronic device 110 may provide a virtual reality service or an augmented reality service independently, or may provide a virtual reality service or an augmented reality service in conjunction with the aerosol generating device 120 or a third electronic device (e.g., a smartphone) wirelessly or wiredly connected to the electronic device 110. The structure and operation method of the electronic device 110 will be described in detail with reference to FIG. 2 below.

[0035] Virtual reality or augmented reality is considered to mean visual simulation that creates a virtual or augmented visual environment. As used herein, the term "virtual reality" includes simulated images that are generated for visualization by the smoker and may include a partially or fully simulated environment. The term "augmented reality" as used herein is also considered to include a combination of simulated images used to augment a real or "live" environment, where the live image is used in combination with a simulated image overlaid on the live image. As a result of virtual or augmented reality, a simulated virtual image can be provided or a live visual image can be augmented with a simulated augmented image to form an interactive environment that can be seen and felt with reality. In the following, for convenience of explanation, the space in which virtual reality or augmented reality is provided will be referred to as a virtual space.

[0036] According to an embodiment, the electronic device 110 is worn at a predetermined position (e.g., on the head) of a user and can provide (or output) an image to the user. For example, the electronic device 110 can be configured in the form of at least one of glasses, goggles, a helmet, or a hat, but is not limited thereto.

[0037] According to an embodiment, the aerosol generating device 120 and the electronic device 110 may be connected via a dongle 130. For example, the aerosol generating device 120 and the dongle 130 may be a pair. A user may connect the dongle 130 and the electronic device 110 by inserting the dongle 130 into an interface (e.g., a universal serial bus (USB) port) of the electronic device 110. The dongle 130 may receive power for operation from the electronic device 110. A wireless communication channel may be established between the dongle 130 and the aerosol generating device 120. For example, the wireless communication channel may be a wireless communication channel based on Bluetooth low energy (BLE), LoRa, EnOcean, radio frequency (RF) communication, Zigbee (registered trademark), or infrared communication.

[0038] According to an embodiment, the electronic device 110 can install a communication driver and a virtual smoking program for the dongle 130 using data stored in the dongle 130. The electronic device 110 and the dongle 130 can transmit and receive information via the communication driver. The electronic device 110 can execute the virtual smoking program to generate and output a virtual smoke image.

[0039] According to one embodiment, an existing application (eg, a virtual world experience) being executed by the electronic device 110 and the virtual smoking program may be executed simultaneously or in parallel.

[0040] According to one embodiment, the electronic device 110 may provide a user with a virtual space experience and virtual smoking via a virtual smoking program.

[0041] According to an embodiment, the dongle 130 may receive detection information related to the state of the aerosol generating device 120 from the aerosol generating device 120 via a wireless communication channel. For example, the detection information related to the state of the aerosol generating device 120 may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a part of the user's body), detection value information of the capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of the camera of the aerosol generating device 120, smoking patterns learned using the pressure sensor and buttons of the aerosol generating device 120, camera data and pressure sensor data, and button input information.

[0042] According to an embodiment, the data format of the detection information received by the dongle 130 from the aerosol generating device 120 is a wireless signal format, and the data format of the detection information may be converted from the wireless signal format to a USB signal format by the dongle 130. The electronic device 110 may receive the detection information converted into the USB signal format from the dongle 130. After execution of the virtual smoking program, virtual smoke information may be generated based on the smoking information of the detection information received from the aerosol generating device 120. A virtual image may be output through a display of the electronic device 110 so that virtual smoke appears in the virtual image based on the virtual smoke information.

[0043] According to an embodiment, the aerosol generating device 120 and the electronic device 110 may be connected via a wireless communication channel. The wireless communication channel may be at least one of short-range communication and cellular communication. For example, the short-range communication may include at least one of Bluetooth (registered trademark), BLE, or wireless LAN (e.g., WiFi direct). For example, the cellular communication may include at least one of LTE (long term evolution), LTE-A (LTE Advance), 5G (or NR (new radio)), CDMA (code division multiple access), WCDMA (wideband CDMA) (registered trademark), or GSM (global system for mobile communication) (registered trademark). Alternatively, the aerosol generating device 120 and the electronic device 110 may be connected by wire based on a USB method. In the above embodiment, the virtual smoking program may be pre-stored in the electronic device 110.

[0044] The system 100 according to an embodiment may further include a playback device 140. For example, the playback device 140 is a device capable of providing sound to the user 10, and the method of providing the sound is not limited. The electronic device 110, the aerosol generating device 120, and the playback device 140 included in the system 100 work together to provide an improved smoking experience to the user 10 using the system 100. Although the electronic device 110 and the playback device 140 are illustrated as separate entities in FIG. 1, the electronic device 110 may include the playback device 140 according to an embodiment.

[0045] According to an embodiment, the electronic device 110 may include one or more auxiliary devices, such as an electroencephalogram (EEG) measuring module, a surface electromyogram (SEM) sensor, a camera, or a haptic glove, in order to enhance the realism of the smoking act in the virtual space.

[0046] FIG. 2 is an exemplary diagram of a display output screen of an electronic device in accordance with one embodiment.

[0047] The description with reference to Fig. 1 may be equally applied to the description with reference to Fig. 2, and duplicated content may be omitted. Hereinafter, the structure and operation method of the electronic device 110 will be described in detail.

[0048] The electronic device 110 according to an embodiment may include a communication unit, a processor, a memory, a display, and a USB port. The electronic device 110 may be a device for virtual reality, augmented reality, or a metaverse, but the present embodiment is not limited thereto. The electronic device 110 may be a device such as a smartphone or a tablet.

[0049] The communication unit is connected to the processor and the memory to transmit and receive data. The communication unit may be connected to another external device to transmit and receive data. Hereinafter, an expression of transmitting and receiving "A" may indicate transmitting and receiving "information or data indicating A".

[0050] The communication unit may be realized as a circuitry within the electronic device 110. For example, the communication unit may include an internal bus and an external bus. As another example, the communication unit may be an element that connects the electronic device 110 to an external device (e.g., the dongle 130). The communication unit may be an interface. The communication unit may receive data from the external device (e.g., the dongle 130) and transmit the data to the processor and the memory.

[0051] The processor processes data received by the communication unit and data stored in the memory. A "processor" may be a data processing device implemented in hardware having circuits with a physical structure for performing desired operations. For example, the desired operations may include code or instructions included in a program. For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an application-specific integrated circuit (ASIC), or a field programmable gate array (FPGA).

[0052] The processor executes computer readable code (e.g., software) stored in the memory and instructions triggered by the processor, for example, the processor may execute instructions that cause a virtual smoke image to be output via a display.

[0053] The memory stores data received by the communication unit and data processed by the processor. For example, the memory may store a program, application, or software (e.g., a communication driver and a virtual smoking program). The stored program may be a set of syntax that is coded to output a virtual smoke image and is executable by the processor.

[0054] According to one embodiment, the memory may include one or more of volatile memory, non-volatile memory, and Random Access Memory (RAM), flash memory, a hard disk drive, and an optical disk drive.

[0055] The memory stores a set of instructions (e.g., software) that causes the electronic device 110 to operate. The set of instructions that cause the electronic device 110 to operate are executed by the processor.

[0056] According to one embodiment, the electronic device 110 and the aerosol generating device 120 may be connected via a dongle 130. For example, the dongle 130 may be connected to a USB port of the electronic device 110. A communication driver and a virtual smoking program stored in the dongle 130 may be installed in the electronic device 110 via the dongle 130.

[0057] According to one embodiment, when the dongle 130 is coupled (or inserted or connected) to a USB port of the electronic device 110, the communication driver and the virtual smoking program may be automatically installed in the electronic device 110. For example, the communication driver and the virtual smoking program may be installed or stored in the memory.

[0058] According to one embodiment, after the dongle 130 is connected to the USB port of the electronic device 110, if a separate input is received from the user, a communication driver and a virtual smoking program may be installed.

[0059] According to one embodiment, the processor can execute a virtual smoking program. For example, the virtual smoking program can be automatically executed after the virtual smoking program is installed. As another example, the virtual smoking program can be executed when a smoking start signal is received from a user or a smoking start signal is received from the aerosol generating device 120. The processor can generate virtual smoke information based on smoking information of the detection information regarding the state of the aerosol generating device 120 received from the aerosol generating device 120 via the dongle 130.

[0060] The electronic device 110 may provide a virtual reality, augmented reality, or metaverse image in which tobacco smoke is generated to a user through a display. In one embodiment, the display may be controlled to be transparent or opaque depending on the intended use. When the electronic device 110 provides a virtual reality or metaverse image, the display may be controlled to be opaque, and when the electronic device 110 provides an augmented reality image, the display may be controlled to be transparent.

[0061] For example, the display may be made of, but is not limited to, a plastic material such as polycarbonate, or a glass material, and may be coated with at least one of a light reflective and anti-glare coating, an anti-fog coating, and an ultraviolet blocking coating.

[0062] According to an example, the system 100 may further include an article. The electronic device 110 may receive content information related to the article from the article. The electronic device 110 may change an image displayed on the display based on the content information related to the article.

[0063] In one embodiment, an image showing tobacco smoke may be displayed on the display, and the electronic device 110 may change the image relating to at least one of the color of the tobacco smoke, the amount of tobacco smoke, and the pattern of the tobacco smoke displayed on the display based on the content information relating to the item.

[0064] The electronic device 110 may further include a camera (not shown) that can scan the actual surroundings when the user uses the electronic device 110 to assist the user in using the electronic device 110.

[0065] The electronic device 110 may be installed with applications to provide a variety of experiences to the user. For example, a smoking application may be installed to provide a virtual space smoking system to provide a user with a smoking experience close to reality. However, the present disclosure is not limited thereto.

[0066] The electronic device 110 may further include a sensor for detecting or predicting a user's smoking behavior in the system 100. For example, the sensor may include a surface electromyogram sensor or an electroencephalogram measuring module attached to the user's skin to detect the user's smoking behavior. The electronic device 110 may include a communication device for communicating with the article. However, the present disclosure is not limited thereto.

[0067] Referring to FIG. 2, a virtual image 200 in which an avatar 210 corresponding to a user 10 of the electronic device 110 and virtual smoke appear is output in a virtual space through a display of the electronic device 110.

[0068] In one embodiment, the virtual image 200 may be output in such a way that virtual smoke appears due to virtual smoking by the avatar 210 through the aerosol generating device 212 in response to the user 10's smoking behavior through the aerosol generating device 120.

[0069] In one embodiment, the movement of the hand 214 of the avatar 210 holding the aerosol generating device 212, i.e., the target movement, can be controlled to appear as a reference movement determined based on detection information of the aerosol generating device 120 of the user 10.

[0070] In one embodiment, the movement of the hand 213 of the avatar 210 that is not holding the aerosol generating device 212, i.e., the target movement, can be controlled to appear as a reference movement determined based on detection information from an auxiliary device such as a camera or haptic glove.

[0071] In one embodiment, a virtual image 200 may be output in which multiple avatars 210, 220 appear.

[0072] In one embodiment, the electronic device 110 may receive information about the second avatar 220 of the second user (e.g., detection information configured to control a target movement of the second avatar 220) from a server coupled to the electronic device 110. For example, the server may provide a virtual space to which multiple users can connect, and the users may connect to the virtual space through their electronic devices. The users' avatars 210, 220 may be placed in the virtual space, and the movements of the avatars 210, 220 appearing through virtual images may be controlled based on detection information from the users' aerosol generating devices (or haptic gloves).

[0073] In one embodiment, the electronic device 110 can receive information about the virtual space from a server and output the second avatar 220 to appear in the virtual space.

[0074] In one embodiment, from the first-person perspective of user 10, a virtual image may be output using the display of electronic device 110 so that the avatars of other users appear except for avatar 210 of user 10. In this case, the hands, feet, virtual smoke, etc. of avatar 210 of user 10 may still be output from the first-person perspective.

[0075] Hereinafter, a method for providing a virtual image to a user will be described in detail with reference to FIGS.

[0076] FIG. 3 is a diagram illustrating a user's smoking behavior divided into sections and a method for predicting a user's exhalation time and outputting a virtual smoke image according to an embodiment.

[0077] The description with reference to FIG. 1 may be equally applied to the description with reference to FIG. 3, and duplicate content may be omitted.

[0078] In one embodiment, a user 1 From time t 4 One smoking session will be performed throughout the process up to this point.

[0079] More specifically, referring to FIG. 3, a user according to one embodiment may 1 From time t 2 D during the time 1 During this period, smokable material (e.g., aerosol) can be inhaled through the aerosol generating device 120. The aerosol generating device 120 can generate aerosol in a variety of ways. 1 The user's action during the period may be referred to as a "smoke inhalation" action, in which the user bites the aerosol generating device 120 in the mouth and inhales the smokable material to collect it in the mouth. 1 In this section, the user and one side of the aerosol generating device 120 (e.g., the inlet or mouthpiece) may be in contact with each other.

[0080] In one embodiment, a user 2 From time t 3 D during the time 2 In the section, the smoke that has gathered in the mouth can be inhaled by inhalation. 2 The user's action in the section can be referred to as an "inhale" action (or an "inhale and lower hand" action), in which the user inhales while separating one side of the aerosol generating device 120 from the lips and lowering the hand. 2 In the section, the user and the aerosol generating device 120 may be separated.

[0081] In one embodiment, a user 3 From time t 4 D between points 3 In the section, the smoke inhaled by exhaling can be expelled. 3 The user's action during the interval may be referred to as an "exhale" action.

[0082] D according to one embodiment 2 The length of the section (inhalation time) is D 1It can be estimated based on the length of the interval (smoke inhalation time). 2 The length of the section (inhalation time) is D 1 It can be proportional to the length of the section (smoke inhalation time). For example, if the smoke inhalation time is the 1st, the inhalation time can also be the 1st, and if the smoke inhalation time is the 3rd, the inhalation time can also be the 3rd. Alternatively, an artificial neural network can be used to predict the inhalation time corresponding to the smoke inhalation time. For example, the smoke inhalation time can be input to an artificial neural network trained based on paired learning data of the smoke inhalation time and the inhalation time, and the corresponding inhalation time can be predicted. However, the method of estimating the inhalation time based on the smoke inhalation time is not limited to the above examples.

[0083] D 2 Being able to predict the length of the interval (inhalation time) means that t 3 This means that the time of expiration can be predicted. 1 D by measuring the length of the section (smoke intake time) 2 The length of the interval can be predicted and the measured D 1 The predicted length of the interval is D 2 Adding the length of the interval gives t 3 (time of expiration) can be predicted.

[0084] According to one embodiment, D 3 The length of the section (expiration time) is D 1 Length of the section (smoke intake time) and D 2 The time period can be determined based on at least one of the following: 1 Length of section or D 2 D is proportional to the length of the section 3 The length of the interval can be determined. 3 (at the time of expiration) and D 3 Based on the length of the interval, t 4 The time can be predicted.

[0085] According to one embodiment, the user 3Information regarding the length of the section can be set in advance in the electronic device 110. For example, 1 Length of section or D 2 The ratio to the length of the section can be set by the user. A user who wants to keep the virtual smoke output time relatively long can set the ratio to a large value.

[0086] In the virtual smoking system according to an embodiment, a user can experience virtual reality or augmented reality of smoking by linking smoking in the real world through the aerosol generating device 120 with the electronic device 110. More specifically, the electronic device 110 according to an embodiment can realize virtual smoke in the virtual space at the time when the user exhales real smoke through the aerosol generating device 120, thereby realizing virtual smoke that does not feel foreign.

[0087] A user of the aerosol generating device 120 according to an embodiment of the present invention 1 From D 3 The user repeats the above operation at least once or more to perform a smoking act, and the electronic device 110 or the aerosol generating device 120 can detect or predict the user's tobacco smoke inhalation, inhalation, and exhalation to realize virtual smoke. Hereinafter, "realizing virtual smoke" may include determining at least one of the output time of the virtual smoke, i.e., the exhalation time, and the attributes of the virtual smoke. For example, the attributes of the virtual smoke may include one or more of the size, shape, duration, volume, density, and color of the virtual smoke. This is not limited in the present specification.

[0088] 3, when a user starts smoking according to an embodiment 300, the user inhales smokable material through the aerosol generating device 120 and collects it in the mouth. At this time, the time that the user inhales the tobacco smoke (e.g., D 1 ) is measured 310. After that, when the user finishes inhaling smoke, the user performs an inhalation and lowering action. At this time, the time during which the user inhales and lowers his / her hand (for example, D 2Based on the smoke inhalation time 310, the electronic device 110 or the aerosol generating device 120 can measure 320 the inhalation and hand down time and the exhalation time when the user starts to exhale (e.g., t 3 When the user starts to exhale, the electronic device 110 can predict the time the user will exhale (e.g., D 3 ) virtual smoke may be realized 340 during the exhalation. After a predetermined time has elapsed since the start of the exhalation, the electronic device 110 may terminate 350 the virtual smoke output.

[0089] FIG. 4 is a flowchart of a method for outputting a virtual smoke image according to an embodiment.

[0090] The description with reference to FIGS. 1 to 3 may be equally applied to the description with reference to FIG. 4, and duplicate content may be omitted.

[0091] The following operations 410-450 are performed by the electronic device 110 described above with reference to FIG.

[0092] In operation 410, the electronic device 110 can output a virtual image using a display of the electronic device 110 so that an avatar corresponding to a user of the electronic device 110 appears in a virtual space.

[0093] In one embodiment, the avatar's face may be generated based on a graphical image of the user's face.

[0094] In one embodiment, the avatar's face may be selected by the user from any of the pre-generated avatars.

[0095] In one embodiment, the avatar's facial expression can change based on facial expression information generated by additional sensors, which can include, for example, one or more of an electroencephalography module, a surface electromyography sensor, or a camera.

[0096] The electroencephalogram measuring module according to one embodiment is a device for measuring an electroencephalogram (EEG) signal, and includes electrodes for detecting an electroencephalogram signal, an amplifier for amplifying a weak electroencephalogram signal, an AD converter for converting a measured analog electroencephalogram into a digital signal, etc. When such an electroencephalogram measuring module is attached to the electronic device 110, it is possible to measure the user's electroencephalogram and obtain facial expression information by analyzing the measured electroencephalogram.

[0097] The surface electromyogram sensor according to one embodiment is capable of detecting the movement of a user by analyzing the electrical signals emitted during muscle contraction. By attaching surface electrodes to the skin to measure the electromyogram signal, a signal that is a composite of motor unit action potentials generated in the muscles around the surface electrodes is measured. By utilizing this, facial expression information can be obtained.

[0098] According to an embodiment, facial expression information can be obtained from information obtained by monitoring the user's face using a camera.

[0099] In one embodiment, the electronic device 110 may receive a connection request from the aerosol generation device 120 via the dongle 130. Specifically, the connection request may be received from the aerosol generation device 120 that has established a wireless communication channel with the dongle 130 via the dongle 130 connected to the electronic device 110. For example, the wireless communication channel may be a wireless communication channel based on BLE, LoRa, EnOcean, RF communication, Zigbee (registered trademark), or infrared communication.

[0100] In one embodiment, the electronic device 110 may receive a connection request from the aerosol generation device 120 via a wireless communication channel. For example, the connection request may be received from the aerosol generation device 120 that has directly established a wireless communication channel with the electronic device 110. The wireless communication channel may be at least one wireless communication of short-range communication or cellular communication. As another example, the aerosol generation device 120 and the electronic device 110 may be connected by wire based on a USB method. The channels through which the electronic device 110 is paired and connected to one or more external controllers and the aerosol generation device 120, respectively, may be separate channels. In the above embodiment, the virtual smoking program may be pre-stored in the electronic device 110.

[0101] In one embodiment, the electronic device 110 can receive a connection request between the aerosol generating device 120 and the electronic device 110 directly from a user via a user interface (UI) of the electronic device 110.

[0102] In operation 420, the electronic device 110 may receive detection information related to the state of the aerosol generating device 120 coupled to the electronic device 110. For example, the detection information may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a body part of a user), detection value information of a capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of a camera of the aerosol generating device 120, smoking patterns learned using a pressure sensor and a button of the aerosol generating device 120, camera data and pressure sensor data, and button input information. The detection information of the aerosol generating device 120 will be described in further detail with reference to FIG. 5 below.

[0103] In operation 430, the electronic device 110 may determine a reference movement of the user based on the detection information, and control a target movement of the avatar so that the determined reference movement appears. The reference movement may mean an actual movement of the user's body or a change in the actual movement, and the target movement may mean a movement of the avatar's body depicted on the image or a change in the movement. In one embodiment, the electronic device 110 may determine a reference movement of the user (e.g., holding the aerosol generating device 120) based on detection information (e.g., button input information of the aerosol generating device 120) corresponding to the start of smoking by the user, and control a target movement of the avatar (e.g., taking out the aerosol generating device from a pocket and holding it) so that the determined reference movement appears.

[0104] In one embodiment, the user's reference motion can be determined based on detection information from an orientation change sensor, such as a gyro sensor, an acceleration sensor, or a 3dof / 6dof sensor, of the aerosol generating device 120. For example, the user's reference motion may be a motion in which the arm holding the aerosol generating device 120 is held at a specific angle and at a specific speed. As another example, the user's reference motion may be a motion in which the hand holding the aerosol generating device 120 is brought toward the user's mouth.

[0105] In one embodiment, the user's reference movement may be determined based on detection information from an auxiliary device such as a camera or a haptic glove. For example, the user's reference movement may be determined based on the movement of the user's hand or arm that is not holding the aerosol generating device 120, captured by a camera. As another example, the user's reference movement may be determined based on detection information from a haptic glove worn by the user on a hand that is holding or not holding the aerosol generating device 120. A specific movement such as bending the user's fingers or making a specific gesture by the haptic glove may be determined as the reference movement.

[0106] In one embodiment, the target motion of the avatar can be controlled to appear as a reference motion. For example, the target motion of the avatar may be a motion that causes the avatar's arm holding the aerosol generating device 120 to be held at a particular angle and at a particular speed. For example, the target motion of the avatar may be a motion that causes the avatar's hand holding the aerosol generating device 120 to be brought toward the user's mouth. For example, the target motion of the avatar may be a motion that causes the fingers of the user not holding the aerosol generating device 120 to curl.

[0107] In one embodiment, the target movement of the avatar may be controlled by a selection information input by the user from among preset possible target movement candidates via the electronic device 110. For example, when the user presses a shortcut key or button for "turn head and exhale," the target movement of the avatar may be controlled by the above input.

[0108] In operation 440, the electronic device 110 may generate virtual smoke information based on the smoking information of the detection information.

[0109] In one embodiment, the electronic device 110 may determine attributes of the virtual smoke based on the smoking information of the detection information. For example, the electronic device 110 may determine attributes of the virtual smoke based on at least a part of the detection information described in operation 420 (e.g., distance information and smoking pattern) for virtual smoking, i.e., the smoking information of the detection information. This is not limited in this specification. Hereinafter, the smoking information of the detection information is information related to smoking among the detection information, and may be simply referred to as detection information.

[0110] In one embodiment, the attributes of the virtual smoke may include one or more of the size, shape, duration, volume, density, and color of the virtual smoke, which is not limited herein. The electronic device 110 may generate the virtual smoke information based on the attributes of the virtual smoke.

[0111] At operation 450, the electronic device 110 may output a virtual image such that virtual smoke appears in the virtual image based on the virtual smoke information.

[0112] In one embodiment, the electronic device 110 can determine the exhalation time based on a first time at which the distance between one side of the aerosol generating device 120 and a part of the user's body reaches within a first threshold, as sensed by the aerosol generating device 120, and a second time after the first time at which the distance between one side of the aerosol generating device 120 and a part of the user's body exceeds a second threshold.

[0113] In one embodiment, the electronic device 110 can determine the exhalation time based on a first time at which the capacitance value of the aerosol generating device 120 sensed by the aerosol generating device 120 reaches within a first threshold and a second time after the first time at which the capacitance value of the aerosol generating device 120 exceeds a second threshold.

[0114] In one embodiment, a virtual image may be output such that virtual smoke appears at the determined exhalation point. The virtual smoke appears based on the virtual smoke information.

[0115] In one embodiment, the target movement of the avatar is controlled and a virtual image can be outputted such that the avatar exhales and emits virtual smoke, which appears based on the virtual smoke information.

[0116] Even if a user cannot visually observe smoke in reality, the user can observe smoke through an image in which virtual smoke appears, and can satisfy the sensation of smoking through the observed smoke.

[0117] FIG. 5 is a diagram illustrating an aerosol generating device according to an embodiment.

[0118] Referring to Fig. 5, the aerosol generating device 500 (e.g., the aerosol generating device 120 of Fig. 1 or the aerosol generating device 120 of Fig. 3) can have a cigarette shape. For example, the length and diameter of the aerosol generating device 500 can be substantially the same as the length and diameter of a traditional cigarette-type cigarette, but are not limited to the described embodiment. The aerosol generating device 500 can be divided into a first end 510, a middle portion 520, and a second end 530.

[0119] The first end 510 can be a portion for replicating the lighting portion of a cigarette. The first end 510 can include an LED. For example, the first end 510 can include a red LED.

[0120] The second end 530 may include nicotine. In one embodiment, the surface of the second end 530 may be coated with nicotine, or the second end 530 may include an aerosol product inside. For example, when the aerosol product is heated, an aerosol containing nicotine may be generated. When a user bites the second end 530 of the aerosol generating device 500 and inhales the aerosol, nicotine may be provided to the user. Meanwhile, the second end 530 may further include a flavoring portion containing a flavoring in addition to the aerosol product.

[0121] In one embodiment, the intermediate portion 520 serves to connect the first end 510 and the second end 530. The length of the intermediate portion 520 determines the overall length of the aerosol generating device 500, and the length of the intermediate portion 520 can be set during manufacturing of the aerosol generating device 500 so that the length of the aerosol generating device 500 corresponds to the length of a typical cigarette.

[0122] In another embodiment, the aerosol generating device 500 may include other electronic devices or electronic circuits inside (e.g., the middle part 520) as an electronic device. For example, the aerosol generating device 500 may include a sensor for acquiring detection information regarding the state of the aerosol generating device 500 or detecting a user's smoking behavior. The sensor may include, for example, a sensor for detecting a change in distance, a sensor for detecting a change in orientation, a sensor for detecting a change in capacitance, and a pressure sensor. The aerosol generating device 500 may include other electronic devices outside the housing as an electronic device. For example, the aerosol generating device 500 may include a camera for observing a user's smoking behavior. The aerosol generating device 500 may have buttons inside or outside for a power adjustment function, a smoking pattern learning function, a smoking detection function, or a control function of the electronic device 110. The aerosol generating device 500 may include a communication device for communicating with an XR device (e.g., the electronic device 110 of FIG. 1 or the electronic device 110 of FIG. 3) as an electronic device. However, the present disclosure is not limited to aerosol generating devices that include a sensor, a camera, or a button. Hereinafter, a method for determining an exhalation time using different detection information will be described. The same or similar content may be omitted.

[0123] <Distance information from distance sensor> The distance detection sensor according to an embodiment may be located inside or outside the second end 530. The distance detection sensor according to an embodiment may be an infrared type, a lidar type, an ultrasonic type, or the like. However, the distance detection sensor of the present disclosure is not limited to the above types. Hereinafter, a method for determining an exhalation time when a user starts exhaling using the distance detection sensor according to an embodiment will be described in detail.

[0124] When a user according to an embodiment starts smoking, the distance detection sensor can detect the distance between an object (e.g., the user's lips) and the aerosol generating device 500. The aerosol generating device 500 can transmit detection information of the distance detection sensor to the electronic device 110 (or the electronic device 110) via a dongle (e.g., the dongle 130 in FIG. 1 or the dongle 130 in FIG. 3). The electronic device 110 can determine the time of exhalation based on the detection information of the distance detection sensor received from the aerosol generating device 500.

[0125] In one embodiment, the act of starting to smoke includes an act of the user bringing the aerosol generating device 500 to their lips and inhaling. The user can turn on the aerosol generating device 500 to start heating the aerosol product in the aerosol generating device 500, and the turned-on aerosol generating device 500 can preheat a heater that heats the aerosol product.

[0126] According to an embodiment, the operation of the distance detection sensor detecting the distance between the object (e.g., a body part of a user) and the aerosol generating device 500 may include an operation of detecting a first time when the object contacts the second end 530 of the aerosol generating device 500 and an operation of detecting a second time when the object moves away from the second end 530 of the article. For example, the operation of detecting the first time may include the distance detection sensor detecting that the second end 530 of the aerosol generating device 500 and the object approach each other to a distance within a first threshold. For example, the operation of detecting the second time may include the distance detection sensor detecting that the object moves away from the second end 530 of the aerosol generating device 500 to a distance exceeding a second threshold after the object contacts the second end 530 of the aerosol generating device 500.

[0127] According to an embodiment, the operation of determining an exhalation time when a user starts exhaling includes an operation of determining an exhalation time based on an interval between a first time and a second time. The first time may be a time when the user starts smoke inhalation. The interval between the first time and the second time is a time when the user takes smoke inhalation, and a first breathing time of the user may be predicted based on the smoke inhalation time. The second time may be a time when the user starts inhalation. A second breathing time may be predicted based on the interval between the first time and the second time and the first breathing time. The second breathing time may be an exhalation time of the user. For example, the second breathing time may be determined as an exhalation time.

[0128] According to an embodiment, since the smoke inhalation time of the user is the interval between the first time and the second time, the first breathing time, which is the time when the user inhales, can be estimated similarly to the smoke inhalation time. Thereafter, the electronic device 110 can determine the time of the second breathing, which is the time when the user starts exhaling, as the time when the first breathing time has elapsed from the second time. For example, if the time from the first time when the object arrives at the second end 530 to the second time when the object is removed from the second end 530 is 1 second, the first breathing time can be estimated to be 1 second, and the time when the user inhales can also be predicted to be 1 second similarly to the first breathing time. As a result, the time of the second breathing, which is the time when the first breathing time has elapsed from the second time, can be predicted to be the time when 1 second has elapsed since the object was separated from the aerosol generating device 500. As a result, the electronic device 110 can determine the time when 1 second has elapsed since the object was removed as the time of exhalation.

[0129] According to an embodiment, the operation of determining the exhalation time point based on the distance detection information includes an operation of estimating the inhalation amount and the respiration amount of the user based on the interval between the first time point and the second time point. The inhalation amount of the user may include the inhalation amount of the aerosol inhaled by the user through the aerosol generating device 500 in a state where the aerosol generating device 500 is in contact with an object. The respiration amount of the user may include the inspiration amount inhaled by the user in a state where the aerosol generating device 500 is not in contact with an object. The exhalation time point may be determined based on the estimated respiration amount and the inhalation amount.

[0130] The operation of determining the time of exhalation based on the detection information according to an embodiment includes an operation of determining whether the aerosol generating device 500 satisfies a predetermined operating condition, and an operation of determining the time of exhalation based on the determination that the aerosol generating device 500 satisfies the operating condition. The predetermined operating condition includes at least one of whether the user turns on / off the power of the aerosol generating device 500, and whether a puff sensor and a pressure sensor included in the aerosol generating device operate. Although the detection value of the distance detection sensor may change depending on other objects (e.g., a finger, etc.) instead of the predetermined object, the distance detection sensor can be activated only when the user turns on / off the power, and the distance detection sensor can be activated by detecting the inhalation of the user through a puff sensor or a pressure sensor, etc. However, the inhalation sensor of the present disclosure is not limited to a puff sensor or a pressure sensor.

[0131] The operation of transmitting the detection information to the electronic device 110 via the dongle 130 according to an embodiment may include an operation of transmitting information on the type of virtual smoke and an operation of transmitting information on a delay time (delay) occurring in communication between the aerosol generating device 500 and the electronic device 110. The type of virtual smoke may be changed according to a user's selection, or may be changed by an aerosol product or other flavoring unit built into the second end 530. The detection information may also include information on the user's smoking amount, breathing amount, etc., and the present disclosure is not limited thereto. If the exhalation time does not match the user's actual smoking behavior or an accurate predicted time cannot be grasped, additional operations may be performed to perform a separate "exhalation prediction detection" processing process.

[0132] <Detection value information of capacitance sensor> The capacitance sensor according to an embodiment may be located inside or outside the second end 530. The capacitance sensor according to an embodiment may be a sensor based on a capacitor. However, the capacitance sensor of the present disclosure is not limited to the above method. Hereinafter, a method for determining an exhalation time when a user starts exhaling using the capacitance sensor according to an embodiment will be described in detail.

[0133] When a user according to an embodiment starts smoking, the capacitance sensor may detect the capacitance of the capacitance sensor, which changes when an object (e.g., the user's lips) comes into contact with the aerosol generating device 500. For example, the capacitance sensor may detect a change in capacitance. The aerosol generating device 500 may transmit the detection information of the capacitance sensor to the electronic device 110 via the dongle 130. The electronic device 110 may determine the time of exhalation based on the detection information of the capacitance sensor received from the aerosol generating device 500.

[0134] In one embodiment, the act of starting smoking includes an act of the user bringing the aerosol generating device 500 into contact with the lips and inhaling. The user can turn on the aerosol generating device 500 to start heating the aerosol product in the aerosol generating device 500, and the turned-on aerosol generating device 500 can preheat a heater that heats the aerosol product.

[0135] According to an embodiment, the operation of detecting a change in capacitance when the object comes into contact with the aerosol generating device 500 by the capacitance sensor may include an operation of detecting a first time when the object comes into contact with the second end 530 of the aerosol generating device 500 and an operation of detecting a second time when the object moves away (or is removed) from the second end 530 of the article. For example, the operation of detecting the first time may include the capacitance sensor detecting that the capacitance detected when the object comes into contact with the second end 530 of the aerosol generating device 500 reaches within a first threshold. For example, the operation of detecting the second time may include the capacitance sensor detecting that the capacitance detected when the object comes into contact with the second end 530 of the aerosol generating device 500 and then is removed exceeds a second threshold.

[0136] The operation of the electronic device 110 according to an embodiment to determine the exhalation time based on the capacitance detection information includes an operation of determining the exhalation time based on an interval between a first time and a second time. The first time may be a time when the user starts inhaling smoke. The interval between the first time and the second time is a time when the user inhales smoke, and a first breathing time of the user may be predicted based on the smoke inhalation time. The second time may be a time when the user starts inhaling. A second breathing time may be predicted based on the interval between the first time and the second time and the first breathing time. The second breathing time may be a time when the user starts exhaling. For example, the second breathing time may be determined as the exhalation time.

[0137] According to an embodiment, since the smoke inhalation time of the user is the interval between the first time and the second time, the first breathing time, which is the time when the user inhales, can be estimated similarly to the smoke inhalation time. Thereafter, the electronic device 110 can determine the time of the second breathing, which is the time when the user starts exhaling, as the time when the first breathing time has elapsed from the second time. For example, if the time from the first time when the object arrives at the second end 530 to the second time when the object is removed from the second end 530 is 1 second, the first breathing time can be estimated as 1 second, and the time when the user inhales can also be predicted as 1 second similarly to the first breathing time. As a result, the time of the second breathing, which is the time when the first breathing time has elapsed from the second time, can be predicted as the time when 1 second has elapsed since the object was separated from the aerosol generating device 500. As a result, the electronic device 110 can determine the time when 1 second has elapsed since the object was removed as the time of exhalation.

[0138] According to an embodiment, the operation of determining the exhalation time point based on the capacitance detection information includes an operation of estimating the inhalation amount and the breathing amount of the user based on the interval between the first time point and the second time point. The inhalation amount of the user may include the inhalation amount of the aerosol inhaled by the user through the aerosol generating device 500 in a state where the aerosol generating device 500 is in contact with an object. The breathing amount of the user may include the inhalation amount of the user inhaled in a state where the aerosol generating device 500 is not in contact with an object. The exhalation time point may be determined based on the estimated breathing amount and the inhalation amount.

[0139] The operation of the electronic device 110 according to an embodiment of the present invention for determining the time of exhalation based on the capacitance detection information includes an operation for determining whether the aerosol generating device 500 satisfies a predetermined operating condition, and an operation for determining the time of exhalation based on the determination that the aerosol generating device 500 satisfies the operating condition. The predetermined operating condition includes at least one of whether the user turns on / off the power of the aerosol generating device 500, and whether the puff sensor and the pressure sensor included in the aerosol generating device operate. Although the detection value of the capacitance sensor may change depending on other objects (e.g., a finger, etc.) instead of the predetermined object, the capacitance sensor can be activated only when the user turns on / off the power, and the capacitance sensor can be activated by detecting the user's inhalation via a puff sensor or a pressure sensor, etc. However, the inhalation sensor of the present disclosure is not limited to a puff sensor or a pressure sensor.

[0140] The operation of transmitting the detection information of the capacitance sensor to the electronic device 110 via the dongle 130 according to an embodiment includes an operation of transmitting information about the type of virtual smoke and an operation of transmitting information about a delay time occurring in communication between the aerosol generating device 500 and the electronic device 110. The type of virtual smoke may be changed according to a user's selection, or may be changed by an aerosol product or other flavoring unit built into the second end 530. The detection information may also include information about the user's smoking amount, breathing amount, etc., and the present disclosure is not limited thereto.

[0141] <Orientation change information> The orientation change sensor according to an embodiment may be located inside or outside the second end 530. The orientation change sensor according to an embodiment may be a sensor such as a gyro sensor, an acceleration sensor, or a 3dof / 6dof sensor. However, the orientation change sensor of the present disclosure is not limited to the above method. Hereinafter, a method for determining an exhalation time when a user starts exhaling using the orientation change sensor according to an embodiment will be described in detail.

[0142] When a user in one embodiment starts smoking, the orientation change sensor can detect an orientation change between an object (e.g., the user's lips) and the aerosol generating device 500. The aerosol generating device 500 can transmit the detection information of the orientation change sensor to the electronic device 110 via the dongle 130. The electronic device 110 can determine the exhalation time based on the detection information of the orientation change sensor.

[0143] According to an embodiment, the operation of the orientation change sensor detecting the orientation change between the object and the aerosol generating device 500 may include an operation of detecting a first time when the object contacts the second end 530 of the aerosol generating device 500 and an operation of detecting a second time when the object moves away from the second end 530 of the article. For example, the operation of detecting the first time may include the orientation change sensor detecting that the second end 530 of the aerosol generating device 500 and the object approach each other in an orientation within a first threshold, and detecting a change in acceleration while the user brings the aerosol generating device 500 with the object. For example, the operation of detecting the second time may include detecting that the object moves away from the object in an orientation exceeding a second threshold after the object contacts the second end 530 of the aerosol generating device 500, and detecting a change in acceleration while the user removes the aerosol generating device 500 from the object.

[0144] According to an embodiment, the orientation change can be measured by comparing the x, y, and z-axis orientations of the aerosol generating device 500 when the user lowers his / her hand with the x, y, and z-axis orientations of the aerosol generating device 500 when the user brings the aerosol generating device 500 into contact with an object. When the user moves his / her hand to move the aerosol generating device 500, the measured acceleration can show a pattern of increasing, decreasing, and stopping. The orientation change can be predicted by measuring the change section of the acceleration, and the user's smoking time can be estimated based on the predicted orientation change.

[0145] According to an embodiment, the electronic device 110 determines the time of expiration based on the detection information of the orientation change, and determines the time of expiration based on the interval between the first time and the second time. Hereinafter, the process of determining the time of expiration is the same as that using the capacitance sensor, and therefore, a description thereof will be omitted.

[0146] <Camera monitoring information> The camera according to an embodiment may be located on the surface of the housing of the aerosol generating device 500. Hereinafter, a method for determining the exhalation time when a user starts exhaling using the camera according to an embodiment will be described.

[0147] When a user in one embodiment starts smoking, the camera can monitor the movement of an object (e.g., the user's lips). The aerosol generating device 500 can transmit the camera monitoring information (i.e., detection information) to the electronic device 110 via the dongle 130. The electronic device 110 can determine the exhalation time based on the camera monitoring information.

[0148] In one embodiment, the operation of the camera monitoring the distance between the object and the aerosol generating device 500 may include an operation of detecting a first time when the motion of the object changes due to contact of the object with the aerosol generating device 500 at the second end 530 of the aerosol generating device 500, and an operation of detecting a second time when the motion of the object changes due to separation of the object from the aerosol generating device 500 from the second end 530 of the article. For example, the operation of detecting the first time may include the camera detecting a first motion of the object bringing the second end 530 of the aerosol generating device 500 to the lips and touching it. For example, the operation of detecting the second time may include the camera detecting a second motion of the object being separated from the second end 530 of the aerosol generating device 500.

[0149] According to an embodiment, the electronic device 110 determines the time of exhalation based on the monitoring information of the camera, and determines the time of exhalation based on the interval between the first time and the second time. Hereinafter, the process of determining the time of exhalation is the same as that using the capacitance sensor, and therefore, a description thereof will be omitted.

[0150] <Smoking patterns learned using pressure sensors and buttons> A pressure sensor and a button (not shown) according to an embodiment may be included in the aerosol generating device 500. The pressure sensor may be located inside or outside the second end 530. The pressure sensor may measure the pressure generated when the user brings the second end to the lips or bites it to smoke, and may detect whether the user is inhaling. The button may have a function of turning on / off the power of the aerosol generating device 500, and may include an input function for storing the user's smoking pattern. The button may include a function of determining whether the user's input is an operation of the power function or an operation for storing the smoking pattern. For example, the button may have a function of turning on / off the power when pressed for a long time, a function of storing the smoking pattern when pressed for a short time, or a function of transmitting detection information such as changing the type of virtual smoke. However, the button of the present disclosure is not limited to the above functions.

[0151] According to an embodiment, when a user starts smoking, the aerosol generating device 500 can learn the smoking pattern using the pressure sensor and the button. The electronic device 110 can determine the exhalation time based on the learned smoking pattern and determine the volume and concentration of the virtual smoke. The aerosol generating device 500 can transmit the smoking pattern (i.e., the detection information) to the electronic device 110 via the dongle 130.

[0152] An operation of learning a smoking pattern using a pressure sensor and a button in one embodiment may include an operation of detecting a first time when an object contacts the second end 530 of the aerosol generating device 500 and the pressure detection value of the pressure sensor is measured to exceed a first threshold, an operation of detecting a second time after the first time when the object is separated from the second end 530 and the pressure detection value of the pressure sensor is measured to be within the second threshold, and an operation of detecting a third time after the second time when the button of the aerosol generating device 500 is operated.

[0153] In the method of learning a smoking pattern according to an embodiment, when a user starts puffing to smoke, the time when the pressure changes by the pressure sensor is measured and stored. Then, the time when the detection value of the pressure sensor is restored at the time when the user stops puffing is stored. The time duration between the start and end of puffing is the smoke inhalation time of the user, which is similar to the user's inhalation time. Then, when the user exhales the smoke he or she has inhaled, the user presses a button of the aerosol generating device 500 to input that exhalation is to begin. When the user's exhalation ends, the user stops pressing the button. The time difference between the start and end of exhalation is similar to the user's exhalation time. The aerosol generating device 500 can generate an algorithm for the user's smoking pattern based on the above-mentioned smoking pattern of the user performed at least once, and can store the smoking pattern learned based on the algorithm.

[0154] In one embodiment, storing a smoking pattern includes storing the smoking pattern based on intervals between a first time and a second time and between the second time and a third time, and collecting data regarding at least one or more intervals between the first time and the second time and between the second time and the third time.

[0155] The operation of determining the time of exhalation based on the learned smoking pattern by the electronic device 110 according to an embodiment includes an operation of determining the time of exhalation based on a period between a first time and a second time. Hereinafter, the process of determining the time of exhalation is the same as that using a capacitance sensor, and therefore a description thereof will be omitted.

[0156] In one embodiment, determining the volume and density of virtual smoke based on the learned smoking pattern may include storing data comparing the pressure sensor operation time with the button operation time, and storing the volume and density of virtual smoke determined based on the compared data in the learned smoking pattern.

[0157] According to an embodiment, the volume and density of the virtual smoke may vary depending on the smoking pattern of the user. For example, if the user's exhalation time is shorter than the user's puff time, the volume of the virtual smoke may be increased to produce thicker smoke. If the user's puff time sensed by the pressure sensor is 2 seconds, but the user presses and releases the button for 1.5 seconds, the volume and density of the virtual smoke may be determined to produce more smoke and thicker smoke. That is, since the user's puff time is the smoke inhalation time and the user's button actuation time is the exhalation time, if the user's exhalation time is shorter than the user's smoke inhalation time, the user exhales an equal amount of smoke in a short period of time, and therefore the electronic device 110 should produce a larger amount of smoke and thicker smoke in a short period of time.

[0158] <Camera data and pressure sensor data> According to an embodiment, a pressure sensor may be included in the aerosol generating device 500. The pressure sensor may be located inside or outside the second end 530. The pressure sensor may measure the pressure generated by the user's lips to detect inhalation.

[0159] 3 and 5, when a user according to an embodiment starts smoking, the electronic device 110 receives camera data detecting the user's hand movement from a camera (not shown). The pressure sensor may obtain pressure sensor data when the user bites the second end 530 with the mouth to smoke. The electronic device 110 may predict the time of exhalation based on the camera data and pressure sensor data received via the dongle 130, i.e., the detection information.

[0160] The electronic device 110 according to an embodiment may be installed with an application capable of monitoring the user's hand movements, and thus may detect the user's smoking behavior data through a camera capable of monitoring the user's hand movements. The camera may monitor a situation in which the user brings the aerosol generating device 500 to the mouth to inhale smoke, inhales the smoke into the lungs by inhaling, and then separates the aerosol generating device 500 from the mouth to exhale the smoke. The data received from the camera and the application installed in the electronic device 110 may analyze the user's hand movements as described above to determine the amount of inhalation and the time of exhalation, etc.

[0161] According to an embodiment, the electronic device 110, the camera, and the aerosol generating device 500 operate as follows. The electronic device 110 in which the smoking device application is installed and the aerosol generating device 500 are connected using the dongle 130. When a user brings the aerosol generating device 500 to his / her mouth for smoking, the application of the electronic device 110 can analyze the user's hand movement using an image (e.g., camera data) captured by the camera. If it is determined through the camera image that the aerosol generating device 500 has reached the mouth and the pressure sensor operates (e.g., the measured pressure value is equal to or greater than a threshold), it can be determined that the user has started smoking. For example, the aerosol generating device 500 can transmit a pressure value measured by the pressure sensor to the electronic device 110 via the dongle 130. After the user starts smoking, the user's action of inhaling smoke into the lungs by inhaling, i.e., the action of separating the aerosol generating device 500 from the mouth, can be monitored using the camera image. By analyzing the interval between the start of smoking and the start of inhalation, the inhalation volume and the amount of inhalation can be estimated, and the time of exhalation can be determined based on the estimated inhalation volume and the amount of inhalation.

[0162] According to one embodiment, the operation of determining the expiration time based on the monitoring result, i.e., the detection information, includes an operation of determining the expiration time based on the interval between the first time and the second time. Hereinafter, the process of determining the expiration time is the same as that using the capacitance sensor, and therefore a description thereof will be omitted.

[0163] FIG. 6 is a flow chart of a method for generating virtual smoke information according to one embodiment.

[0164] According to one embodiment, operation 440 described above with reference to FIG. 4 may include the following operations 610 and 620.

[0165] In operation 610, the electronic device 110 can determine attributes of the virtual smoke based on the smoking information of the detection information received from the aerosol generating device 120 of Fig. 1 or the aerosol generating device 500 of Fig. 5. For example, the attributes of the virtual smoke can include one or more of the size, shape, duration, volume, density, and color of the virtual smoke, which is not limited herein.

[0166] In one embodiment, the size, duration, volume, and concentration of the virtual smoke may be determined to be proportional to the smoke inhalation time or the inhalation time based on the smoke inhalation time predicted using one or more of distance information between the aerosol generating device 120 and an object (e.g., a body part of a user), detection value information of the capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of the camera of the aerosol generating device 120, smoking patterns learned using the pressure sensor and buttons of the aerosol generating device 120, camera data, pressure sensor data, and button input information. For example, if the inhalation time is long, the duration of the virtual smoke may be determined to be the same or proportionally longer.

[0167] In one embodiment, the shape of the virtual smoke may be determined by camera monitoring information. When the camera of the electronic device 110 receives information monitoring an object, such as a user's lips, the camera monitoring information may include information about the shape of the lips. For example, if the user's lips are pursed, the shape of the virtual smoke may be determined to be thin and long.

[0168] In one embodiment, the volume, thickness and color of the virtual smoke may be determined by the user's smoking pattern, for example, if the user spends less time exhaling than inhaling smoke, the volume of the virtual smoke may be increased to produce a thicker smoke and the color of the virtual smoke may be determined to be darker.

[0169] In one embodiment, the electronic device 110 can directly receive user input regarding attributes of the virtual smoke. For example, the color of the virtual smoke can be various colors such as blue, purple, etc., depending on the user input, as well as a general neutral color. As another example, the shape of the virtual smoke can be a donut shape.

[0170] In one embodiment, the smoking information of the detection information regarding the state of the aerosol generating device 120 is pressure sensor data, and may include airflow detection information due to the user's inhalation. If the internal flow air pressure where the pressure sensor of the aerosol generating device 120 is installed is high, the virtual smoke may be determined to be large and long lasting even if the inhalation time is short.

[0171] At operation 620, the electronic device 110 can generate virtual smoke information based on the attributes of the virtual smoke. In one embodiment, the virtual smoke information can be generated by further adjusting the pre-defined basic attributes based on the attributes of the virtual smoke determined at operation 610.

[0172] FIG. 7 is a flowchart of a method for outputting a virtual smoke image at the time of exhalation according to an embodiment.

[0173] According to one embodiment, operations 450 described above with reference to FIG. 4 may include the following operations 710 and 720.

[0174] In operation 710, the electronic device 110 determines an exhalation time when the user of the aerosol generating device 120 starts exhaling based on smoking information of the detection information related to the state of the aerosol generating device 120. The smoking information of the detection information related to the state of the aerosol generating device 120 may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a body part of a user), detection value information of the capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of the camera of the aerosol generating device 120, smoking patterns learned using the pressure sensor and button of the aerosol generating device 120, camera data and pressure sensor data, and button input information.

[0175] In operation 720, the electronic device 110 outputs a virtual image of the virtual smoke appearing at the time of exhalation. In one embodiment, the virtual image is output via a display of the electronic device 110.

[0176] In one embodiment, a virtual image may be output in which virtual smoke appears due to virtual smoking through an aerosol generating device of an avatar in response to a user's smoking behavior through the aerosol generating device 120.

[0177] In one embodiment, a virtual image can be output in which multiple avatars appear.

[0178] Even if a user cannot visually observe smoke in reality, the user can observe smoke through an image in which virtual smoke appears, and can satisfy the sensation of smoking through the observed smoke.

[0179] FIG. 8 is a flowchart of a method for outputting a virtual smoke image so that a second avatar appears in a virtual space according to an embodiment.

[0180] According to one embodiment, operation 450 described above with reference to FIG. 4 may include the following operations 810 and 820.

[0181] At operation 810 , the electronic device 110 may receive information regarding a second avatar of a second user from a server coupled to the electronic device 110 .

[0182] According to one embodiment, the information on the second avatar of the second user can be detection information configured to control a target movement of the second avatar, the target movement of the second avatar can be controlled to appear as a reference movement of the second user determined based on the detection information of the aerosol generating device of the second user.

[0183] According to an embodiment, a server can provide a virtual space to which multiple users can connect, and the users can connect to the virtual space through their electronic devices. The users' avatars (e.g., avatars 210 and 220 in FIG. 2) can be placed in the virtual space, and the movements of each of the avatars appearing through the virtual image can be controlled based on detection information of the user's aerosol generating device (or haptic glove). For example, the electronic device 110 can receive information about the virtual space in which the avatars are placed from the server, and information about the movements of a second avatar from the server.

[0184] At operation 820, the electronic device 110 may output a virtual image such that the second avatar appears in the virtual space.

[0185] According to one embodiment, the electronic device 110 can control a target movement of the second avatar based on information about the second avatar, and generate virtual smoke information for the second avatar, thereby outputting the second avatar to appear in a virtual space.

[0186] According to one embodiment, the electronic device 110 can receive information about a second avatar to which the target movement control and the generated virtual smoke information are applied from a server connected to the electronic device 110, and output a virtual image so that the second avatar appears in a virtual space.

[0187] The above-described embodiments may be implemented with hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications executed on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, although it has been described that one processing device is used, a person skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0188] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure or instruct the processing device to operate as desired, either individually or collectively. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may also be distributed over network-coupled computer systems, and stored or executed in a distributed manner. The software and data may be stored in a computer-readable recording medium.

[0189] The method according to the embodiment may be realized in the form of program instructions executed by various computer means and recorded on a computer readable medium. The computer readable medium may store the program instructions, data files, data structures, and the like, alone or in combination, and the program instructions recorded on the medium may be those specially designed or constructed for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as Read Only Memory (ROM), RAM, and flash memory. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, as well as machine code such as produced by a compiler.

[0190] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of an embodiment, and vice versa.

[0191] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations thereto. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

[0192] Therefore, other embodiments, other examples, and equivalents to the claims are within the scope of the following claims.

Claims

1. A method for outputting a virtual smoke image by an electronic device, comprising: outputting a virtual image using a display of the electronic device so that an avatar corresponding to a user of the electronic device appears in a virtual space; receiving detection information relating to a state of an aerosol generating device coupled to the electronic device; determining a reference movement of the user based on the detection information, and controlling a target movement of the avatar so that the determined reference movement appears; generating virtual smoke information based on the smoking information of the detection information; an operation of outputting the virtual image based on the virtual smoke information so that virtual smoke appears in the virtual image; Including, Virtual video output method.

2. the avatar's face is generated based on a graphical representation of the user's face; the facial expression of the avatar changes based on facial expression information generated by an additional sensor; The virtual image output method according to claim 1 .

3. The operation of generating virtual smoke information based on the detection information includes: determining attributes of the virtual smoke based on the smoking information of the detection information, the attributes of the virtual smoke including at least one of a size, a shape, and a duration of the virtual smoke; and generating the virtual smoke information based on the attributes of the virtual smoke; Including, The virtual image output method according to claim 1 .

4. The operation of outputting the virtual image so that virtual smoke appears includes: determining an exhalation time when a user of the aerosol generating device starts exhaling based on the smoking information of the detection information; and outputting the virtual image so that the virtual smoke appears at the time of exhalation; Including, The virtual image output method according to claim 1 .

5. The operation of determining an exhalation time when a user of the aerosol generating device starts exhaling based on the smoking information of the detection information includes: determining the exhalation time based on a first time when a distance between one side of the aerosol generating device and the body part of the user reaches within a first threshold value, and a second time when the distance between the one side of the aerosol generating device and the body part of the user exceeds a second threshold value after the first time; Including, The virtual image output method according to claim 4.

6. The operation of determining an exhalation time when a user of the aerosol generating device starts exhaling based on the smoking information of the detection information includes: determining the exhalation time based on a change in capacitance of the aerosol generating device; Including, The virtual image output method according to claim 4.

7. The operation of determining the exhalation time based on the change in capacitance of the aerosol generating device includes: determining the exhalation time based on a first time at which a value of the capacitance of the aerosol generating device, as sensed by the aerosol generating device, falls within a first threshold value and a second time after the first time at which the value of the capacitance of the aerosol generating device exceeds a second threshold value; Including, The virtual image output method according to claim 6.

8. The detection information is received from the aerosol generating device that has established a wireless communication channel with a dongle connected to the electronic device through the dongle. The virtual image output method according to claim 1 .

9. The detection information is received from the aerosol generating device that directly establishes a wireless communication channel with the electronic device. The virtual image output method according to claim 1 .

10. The operation of outputting a virtual image so that the avatar appears includes: receiving information regarding a second avatar of a second user from a server coupled to the electronic device; and an operation of outputting a virtual image so that the second avatar appears in the virtual space; Including, The virtual image output method according to claim 1 .

11. A program for causing a processor provided in the electronic device to execute the virtual image output method described in any one of claims 1 to 10.

12. 1. An electronic device comprising: A display and A memory in which a program for outputting a virtual smoke image is recorded; A processor for executing the program; Including, The processor, A method for outputting a virtual smoke image by an electronic device, comprising: outputting a virtual image using a display of the electronic device so that an avatar corresponding to a user of the electronic device appears in a virtual space; receiving detection information relating to a state of an aerosol generating device coupled to the electronic device; determining a reference movement of the user based on the detection information, and controlling a target movement of the avatar so that the determined reference movement appears; generating virtual smoke information based on the smoking information of the detection information; an operation of outputting the virtual image based on the virtual smoke information so that virtual smoke appears in the virtual image; To carry out electronic equipment.

Images