Virtual smoke image output method and device

Abstract

In order to output a virtual smoke image, an electronic device performs the steps of: receiving a connection request from an aerosol-generating device in a state of being paired with one or more external controllers; determining a controller type of the aerosol-generating device; when the determined controller type corresponds to a target external controller type in the more than one external controller, pairing with the target external controller is stopped, and the controller is connected with the aerosol generating device; receiving sensing information about a state of the aerosol-generating device; generating a virtual smoke image based on the sensing information; and outputting the virtual smoke image through a display of the electronic device.

Description

Technical Field

[0001] The following embodiments relate to a technology for implementing virtual smoke in an electronic device. Background Art

[0002] Recently, virtual reality, augmented reality and mixed reality technologies that use computer graphics technology have been continuously developed. Here, virtual reality technology refers to the technology of using computers to build a virtual space that does not exist in the real world and then make the virtual space feel real; while augmented reality or mixed reality technology refers to the technology of superimposing computer-generated information on the real world for expression, that is, the technology of interacting with users in real time by combining the real world and the virtual world.

[0003] Among them, augmented reality and mixed reality technologies will be combined with technologies in various fields. In the field of electronic cigarettes, there is an increasing demand for devices that can realize virtual reality or augmented reality services, and related research is very active. Summary of the invention

[0004] Technical issues to be solved

[0005] An embodiment provides a virtual smoke image output method executed by an electronic device.

[0006] An embodiment provides an electronic device for executing a virtual smoke image output method.

[0007] Technical methods to solve problems

[0008] According to one embodiment, a virtual smoke image output method performed by an electronic device may include the following steps: receiving a connection request from an aerosol generating device in a state of being paired with one or more external controllers; determining a controller type of the aerosol generating device; when the determined controller type corresponds to a target external controller type among the one or more external controllers, stopping pairing with the target external controller and connecting with the aerosol generating device; receiving sensing information about the state of the aerosol generating device; generating a virtual smoke image based on the sensing information; and outputting the virtual smoke image through a display of the electronic device.

[0009] The step of outputting the virtual smoke image through the display of the electronic device may include the following steps: superimposing the virtual smoke image on a reference image of an existing application being executed by the electronic device and outputting the result.

[0010] The step of generating a virtual smoke image based on the sensing information may include the following steps: determining properties of virtual smoke based on the sensing information, wherein the properties of the virtual smoke include at least one of the size, shape and duration of the virtual smoke; and generating the virtual smoke image based on the properties of the virtual smoke.

[0011] The step of outputting the virtual smoke image through the display of the electronic device may include the following steps: determining an exhalation moment at which a user of the aerosol generating device starts to exhale based on the sensing information; and outputting the virtual smoke image at the exhalation moment.

[0012] The step of determining the exhalation moment when the user of the aerosol generating device starts to exhale based on the sensing information may include the following steps: determining the exhalation moment based on a first moment and a second moment sensed by the aerosol generating device, wherein the first moment is the moment when the distance between one side of the aerosol generating device and a part of the user's body reaches within a first threshold; and the second moment is the moment after the first moment when the distance between the one side of the aerosol generating device and the part of the user's body exceeds a second threshold.

[0013] The step of determining an exhalation moment at which a user of the aerosol generating device starts to exhale based on the sensed information may comprise the step of determining the exhalation moment based on a change in capacitance of the aerosol generating device.

[0014] The step of determining the exhalation moment based on the change in capacitance of the aerosol generating device may include the following steps: determining the exhalation moment based on a first moment and a second moment sensed by the aerosol generating device, wherein the first moment is the moment when the capacitance value of the aerosol generating device reaches a first threshold value; and the second moment is the moment when the capacitance value of the aerosol generating device exceeds a second threshold value after the first moment.

[0015] The connection request may be received via a dongle connected to the electronic device, from the aerosol generating device establishing a wireless communication channel with the dongle.

[0016] The connection request may be received from the aerosol generating device directly establishing a wireless communication channel with the electronic device.

[0017] According to one embodiment, an electronic device that executes a virtual smoke image output method may include: a display; a memory that records a virtual smoke image output program; and a processor that executes the program, wherein the processor performs the following steps: receiving a connection request from an aerosol generating device in a state of being paired with at least one external controller; determining a controller type of the aerosol generating device; when the determined controller type corresponds to a target external controller type among the at least one external controller, stopping pairing with the target external controller and connecting with the aerosol generating device; receiving sensing information about the state of the aerosol generating device; generating a virtual smoke image based on the sensing information; and outputting the virtual smoke image through the display of the electronic device.

[0018] Effects of the Invention

[0019] The invention can provide a method for outputting a virtual smoke image through a display of an electronic device.

[0020] The present invention may provide a method of using an aerosol generating device as a controller of an electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 2 is a diagram illustrating a virtual smoke image output system according to an embodiment.

[0022] Figure 2 1 is a diagram illustrating a method for classifying a user's smoking behavior according to different stages and a diagram illustrating a method for predicting a user's exhalation moment and outputting a virtual smoke image according to an embodiment.

[0023] Figure 3 2 is a diagram illustrating a virtual smoke image output system according to an embodiment.

[0024] Figure 4 is a flow chart of a method for outputting a virtual smoke image according to an embodiment.

[0025] Figure 5 is an exemplary diagram of an aerosol generating device according to an embodiment.

[0026] Figure 6 is a flow chart of a method for outputting a virtual smoke image according to an embodiment.

[0027] Figure 7 is a flow chart of a method for outputting a virtual smoke image according to an embodiment. DETAILED DESCRIPTION

[0028] The description of the specific structure or function of the embodiment is only for example, and the embodiment can be implemented in different forms. The actual implementation is not limited to the specific embodiment disclosed, and the scope of this specification includes all changes within the technical ideas described by the embodiment, their equivalents and even their substitutes.

[0029] The terms "first" or "second" may be used to describe different components, but are only used to distinguish one component from other components. For example, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0030] When it is stated that one component is “connected to” another component, it may be directly connected to or in contact with the other component, or there may be another component between them.

[0031] In the absence of special instructions in the content, a single expression includes multiple meanings. In this specification, the terms "including" or "having" are used to express the existence of features, numbers, steps, operations, constituent elements, accessories or combinations thereof recorded in the specification, and do not exclude the possibility of the existence or additional addition of at least one other feature, number, step, operation, constituent element, accessory or combination thereof.

[0032] In the absence of other definitions, all terms used herein, including technical or scientific terms, have the common meanings understood by those of ordinary skill in the art. Commonly used terms as defined in dictionaries should be understood as having the meanings in the relevant technical content and should not be interpreted as idealized or overly formalized meanings if they are not clearly defined in this specification.

[0033] In the following, the embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the accompanying drawing numbers, and repeated descriptions are omitted.

[0034] Figure 1 2 is a diagram illustrating a virtual smoke image output system according to an embodiment.

[0035] Reference Figure 1 The virtual smoke image output system 100 includes an electronic device 110 for outputting virtual smoke images, an aerosol generating device 120, an adapter 130, and at least one external controller 150, 160. For example, a user 10 using the system 100 wears the electronic device 110 and controls an existing application (e.g., watching a video, playing a game) executed by the electronic device 110 through at least one external controller 150, 160 paired with the electronic device 110, that is, utilizing the control function of the electronic device 110.

[0036] According to one embodiment, the user 10 using the system 100 may wear the electronic device 110 and smoke through the aerosol generating device 120. "Smoking" may refer to heating, burning, and other behaviors that release smoke or aerosol from a smokable substance. According to one embodiment, the electronic device 110 may realize virtual smoke through images by linking with the smoking behavior of the user 10 using the aerosol generating device 120.

[0037] According to one embodiment, Figure 1 As shown, the user 10 using the system 100 can smoke using the aerosol generating device 120 in a state where the electronic device 110 and at least one external controller 150 , 160 are paired.

[0038] For example, the external controllers 150 and 160 may be referred to as motion controllers, touch controllers, remote controllers, or joysticks. For example, the external controllers 150 and 160 may be used as sensors for sensing the hand or arm movements of the user 10. The external controllers 150 and 160 not only sense the up, down, left, and right movements of the hand, but also sense small movements such as changes in the wrist angle. The external controllers 150 and 160 may include position change sensors such as gyroscope sensors, acceleration sensors, and three-degree-of-freedom / six-degree-of-freedom (3dof / 6dof) sensors. In addition, the external controllers 150 and 160 may also include buttons for receiving user input through fingers.

[0039] In one embodiment, the aerosol generating device 120 may be referred to as an electronic cigarette device, an inhaler, or a cigarette stick. Figure 5 The structure and specific operation method of the aerosol generating device 120 are described in detail.

[0040] According to one embodiment, the user smokes using the aerosol generating device 120 and generates smoke through the user's breathing. According to one embodiment, the aerosol generating device 120 can provide aerosol to the user by heating the aerosol generating product in the aerosol generating device 120 in various ways. For example, the aerosol generating product can be a cigarette stick inserted into the aerosol generating device 120. For another example, the aerosol generating product can be a replaceable cigarette cartridge inserted into the aerosol generating device 120.

[0041] According to an embodiment, the user 10 inhales the aerosol provided by the aerosol generating device 120 and can observe smoke as the user 10 breathes, but the invention is not limited thereto. For example, smoke may not be generated when the user breathes.

[0042] According to one example, a virtual smoke image may be output when the user smokes using the aerosol generating device 120 but does not generate smoke. According to one example, a virtual smoke image may be output even if the user does not actually smoke using the aerosol generating device 120. For example, a virtual smoking program of the electronic device 110 may provide a specific mode to generate and output a virtual smoke image based on sensing information received from the aerosol generating device 120. Therefore, when the user 10 smokes using the aerosol generating device 120, even if the user 10 does not actually observe smoke when exhaling, the user 10 may observe the virtual smoke image output by the electronic device 110 through virtual reality or augmented reality. However, the present disclosure is not limited thereto.

[0043] According to an embodiment, the electronic device 110 may provide a virtual reality service or an augmented reality service alone, or may provide a virtual reality service or an augmented reality service by linking with a third electronic device (e.g., a smart phone) that is connected to the aerosol generating device 120 or the electronic device 110 by wire or wireless. Figure 3 The structure and operation method of the electronic device 110 are described in detail.

[0044] Virtual reality or augmented reality refers to a visual simulation that creates a virtual or enhanced visual environment. The term "virtual reality" herein may include a simulated image of a part or all of a simulated environment that provides visual effects to a smoker. The term "augmented reality" herein may include a combination of simulated images used to enhance a real or "live" environment, wherein a live image is used in combination with a simulated image overlaid on the live image. As a result of virtual reality or augmented reality, a simulated virtual image may be provided, or an enhanced image based on a simulated real-time visual image may be further enhanced, thereby forming an interactive environment that can be truly observed and felt. In the following, for ease of explanation, the space that provides virtual reality or augmented reality is referred to as a virtual space.

[0045] According to one embodiment, the electronic device 110 can be worn at a predetermined position of the user (e.g., head) to provide (or output) images to the user. For example, the electronic device 110 can be configured in at least one form of glasses, goggles, a helmet, or a hat, but is not limited thereto.

[0046] According to one embodiment, at least one external controller 150, 160 may be paired with the electronic device 110 via a wireless communication channel. For example, the wireless communication channel may be a wireless communication channel based on Bluetooth low energy (BLE), LoRa, EnOcean, radio frequency (RF) communication, Zigbee, or infrared communication.

[0047] According to one embodiment, the aerosol generating device 120 and the electronic device 110 may be connected via an adapter (dongle) 130. For example, the aerosol generating device 120 and the adapter 130 may be a pair. The user may connect the adapter 130 and the electronic device 110 by inserting the adapter 130 into an interface (e.g., USB (universal serial bus)) of the electronic device 110. The adapter 130 may receive power required for operation from the electronic device 110. A wireless communication channel may be established between the adapter 130 and the aerosol generating device 120. For example, the wireless communication channel may be a wireless communication channel based on Bluetooth low energy (BLE), LoRa, EnOcean, radio frequency (RF) communication, Zigbee, or infrared communication.

[0048] According to an embodiment, the electronic device 110 can use the data stored in the adapter 130 to install a communication driver and a virtual smoking program for the adapter 130. The electronic device 110 and the adapter 130 can send and receive information through the communication driver. The electronic device 110 can execute the virtual smoking program to generate and output a virtual smoke image.

[0049] According to one embodiment, the existing application program (eg, a game) and the virtual smoking program executed by the electronic device 110 may be executed simultaneously or in parallel.

[0050] According to an embodiment, the adapter 130 may receive sensing information about the state of the aerosol generating device 120 from the aerosol generating device 120 through a wireless communication channel. For example, the sensing information about the state of the aerosol generating device 120 may include distance information between the aerosol generating device 120 and an object (e.g., a part of the user's body), sensing value information of a capacitive sensor of the aerosol generating device 120, position change information of the aerosol generating device 120, camera monitoring information of the aerosol generating device 120, smoking patterns learned using a pressure sensor and a button of the aerosol generating device 120, camera data and pressure sensor data, and at least one of button input information.

[0051] According to an embodiment, the data format of the sensing information received by the adapter 130 from the aerosol generating device 120 is a wireless signal format, and the data format of the sensing information can be converted from the wireless signal format to the USB signal format by the adapter 130. The electronic device 110 receives the sensing information converted into the USB signal format from the adapter 130. After executing the virtual smoking program, a virtual smoke image can be generated based on the sensing information received from the aerosol generating device 120. The virtual smoke image can be output through the display of the electronic device 110.

[0052] According to one embodiment, the aerosol generating device 120 and the electronic device 110 may be connected via a wireless communication channel. The wireless communication channel may be at least one of short-range communication or cellular communication. For example, short-range communication may include at least one of Bluetooth, Bluetooth low energy (BLE), or wireless LAN (e.g., WiFi direct). For example, cellular communication may include at least one of long term evolution (LTE), LTE Advance (LTE-A), 5G (or new radio (NR)), code division multiple access (CDMA), wideband CDMA (WCDMA), or global system for mobile communication (GSM). Optionally, the aerosol generating device 120 and the electronic device 110 may be wired based on a universal serial bus (USB). The channel for pairing the electronic device 110 with at least one external controller 150, 160, and the channel for pairing (or connecting) the electronic device 110 with the aerosol generating device 120 may be different channels. In the above embodiment, the virtual smoking program may be pre-stored in the electronic device 110 .

[0053] According to an embodiment, the system 100 may further include a playback device 140. For example, the playback device 140 may be a device capable of providing audio to the user 10, and the method of providing audio is not limited. By linking the electronic device 110, the aerosol generating device 120 and the playback device 140 in the system 100, a better smoking experience may be provided to the user 10 using the system 100. Figure 1 In the embodiment, the electronic device 110 and the playback device 140 are separate, but according to an embodiment, the electronic device 110 may include the playback device 140.

[0054] According to an embodiment, the electronic device 110 may include at least one auxiliary device. For example, the electronic device 110 may include a brain wave measurement module, a surface electromyography sensor, etc., to improve the realism of smoking behavior in the virtual space.

[0055] Figure 2 1 is a diagram illustrating a method for classifying a user's smoking behavior according to different stages and a diagram illustrating a method for predicting a user's exhalation moment and outputting a virtual smoke image according to an embodiment.

[0056] Reference Figure 1 The same applies to reference Figure 2 The repeated contents are omitted here.

[0057] According to one embodiment, the user completes a smoking behavior through the process from time t1 to time t4.

[0058] More specifically, refer to Figure 2 According to one embodiment, the user can inhale the smokable substance (e.g., aerosol) using the aerosol generating device 120 during the D1 stage between the time t1 and the time t2. The aerosol generating device 120 can generate aerosols in different ways. Hereinafter, the user's action during the D1 stage is referred to as the "inhaling smoke" action, and the user can hold the aerosol generating device 120 in the mouth and inhale the smokable substance into the mouth through the action of inhaling smoke. During the D1 stage, the user and one side of the aerosol generating device 120 (e.g., the inhalation port or the mouthpiece) can be in contact.

[0059] According to an embodiment, the user can inhale the smoke in the mouth by inhaling in the D2 stage between time t2 and time t3. Hereinafter, the user's action in the D2 stage is referred to as the "inhalation" action (or "inhalation and hand-down" action), and the user takes one side of the aerosol generating device 120 away from the mouth and puts the hand down while inhaling. In the D2 stage, the user and the aerosol generating device 120 may be in a separated state.

[0060] According to an embodiment, the user can exhale the inhaled smoke by exhaling in the D3 stage between time t3 and time t4. Hereinafter, the user's action in the D3 stage is referred to as an "exhalation" action.

[0061] According to one embodiment, the length of the D2 stage (inhalation time) can be estimated based on the length of the D1 stage (smoke inhalation time). For example, the length of the D2 stage (inhalation time) can be proportional to the length of the D1 stage (smoke inhalation time). For example, when the smoke inhalation time is 1 second, the inhalation time can also be 1 second, and when the smoke inhalation time is 3 seconds, the inhalation time can also be 3 seconds. Alternatively, an artificial neural network can be used to predict the inhalation time corresponding to the smoke inhalation time. For example, the smoke inhalation time can be input into an artificial neural network trained based on paired learning data of the smoke inhalation time and the inhalation time, thereby predicting the inhalation time corresponding thereto. However, the method of estimating the inhalation time based on the smoke inhalation time is not limited to the above example.

[0062] Being able to predict the length of the D2 phase (inhalation time) means that t3 (exhalation time) can also be predicted. In other words, the length of the D2 phase can be predicted by measuring the length of the D1 phase (smoke inhalation time), and t3 (exhalation time) can be predicted by adding the predicted length of the D2 phase to the measured length of the D1 phase.

[0063] According to an embodiment, the length of the D3 stage (exhalation time) may be determined based on at least one of the length of the D1 stage (smoke inhalation time) and the length of the D2 stage (inhalation time). For example, the length of the D3 stage may be proportional to the length of the D1 stage or the length of the D2 stage. The t4 moment may be predicted based on the predicted t3 (exhalation moment) and the length of the D3 stage.

[0064] According to one embodiment, the user may pre-set the length information of the D3 stage in the electronic device 110. For example, the user may set the ratio of the length of the D1 stage or the length of the D2 stage. If the user wants to keep the virtual smoke realization time for a long time, a large ratio may be set.

[0065] In the virtual smoking system according to an embodiment, the user can experience the virtual reality or augmented reality of smoking by linking the smoking through the aerosol generating device 120 in the real world with the electronic device 110. More specifically, the electronic device 110 according to an embodiment can also realize virtual smoke in the virtual space at the moment when the user actually exhales smoke through the aerosol generating device 120, thereby naturally realizing virtual smoke.

[0066] According to an embodiment, the user of the aerosol generating device 120 smokes by repeating at least one of the actions D1 to D3, and the electronic device 110 or the aerosol generating device 120 can realize virtual smoke by sensing or predicting the user's inhaled smoke, inhalation, and exhalation. Hereinafter, "realizing virtual smoke" may include determining the output moment of the virtual smoke, i.e., the exhalation moment, and at least one of the virtual smoke attributes. For example, the attributes of the virtual smoke may include at least one of the size, shape, duration, volume, concentration, and color of the virtual smoke. This specification is not limited thereto.

[0067] Reference Figure 2 According to an embodiment, in step 200, when the user starts smoking, the user can inhale the smokable material into the mouth through the aerosol generating device 120. At this time, in step 210, the time when the user inhales tobacco smoke can be measured (for example, D1). Thereafter, when the user ends the smoke inhalation, the user will inhale and put down his hand. At this time, in step 220, the time when the user inhales and puts down his hand can be measured (for example, D2). The electronic device 110 or the aerosol generating device 120 can predict the time to inhale and put down the hand and the exhalation moment when the user starts to exhale (for example, t3) based on the smoke inhalation time measured in step 210. In step 240, when the user starts to exhale, the electronic device 110 can realize virtual smoke during the user's exhalation (for example, D3). In step 250, after starting to exhale and after a predetermined time, the electronic device 110 can stop outputting virtual smoke.

[0068] Figure 3 2 is a diagram illustrating a virtual smoke image output system according to an embodiment.

[0069] According to one embodiment, the virtual smoke image output system 300 (eg, Figure 1 The system 100 includes an electronic device 310 (eg, Figure 1 electronic device 110), an aerosol generating device 320 (e.g., Figure 1 aerosol generating device 120), adapter 330 (e.g., Figure 1 adapter 130) and at least one external controller (e.g., Figure 1 For example, the aerosol generating device 320 and the adapter 330 may be a pair. For example, the aerosol generating device 320 may be an inhalation device such as an electronic cigarette or an inhaler.

[0070] According to an embodiment, the electronic device 310 may include a communication unit 311, a processor 312, a memory 313, a display 314, and a USB port 315. The electronic device 310 may be a device for virtual reality, augmented reality, or metaverse, but the embodiment is not limited thereto.

[0071] The communication unit 311 is connected to the processor 312 and the memory 313 to transmit and receive data. The communication unit can be connected to other external devices to transmit and receive data. In the following, transmitting and receiving "A" means transmitting and receiving "information (information) or data representing A".

[0072] The communication unit 311 may be implemented as a circuit within the electronic device 310. For example, the communication unit 311 may include an internal bus and an external bus. For another example, the communication unit 311 may be a component that connects the electronic device 310 to an external device (e.g., an adapter 330, or at least one external controller). The communication unit 311 may be an interface. The communication unit 311 may receive data from an external device (e.g., an adapter 330) and transmit data to the processor 312 and the memory 313.

[0073] The processor 312 processes the data received by the communication unit 311 and the data stored in the memory 313. The "processor" may be a data processing device that is implemented by hardware including a physical structure circuit that performs the desired operations. For example, the desired operations may include a code or instructions contained in a program. For example, a data processing device implemented as hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

[0074] The processor 312 is used to execute computer readable codes (such as software) stored in the memory 313 and instructions issued by the processor 312. For example, the processor 312 can execute instructions to output a virtual smoke image through the display 314.

[0075] The memory 313 stores data received by the communication unit 311 and data processed by the processor 312. For example, the memory 313 can store programs, applications or software (e.g., communication drivers and virtual smoking programs). The stored programs can be encoded as a syntax set that can output virtual smoke images and be executed by the processor 312.

[0076] According to one aspect, the memory 313 may include at least one of a volatile memory, a non-volatile memory, a random access memory (RAM), a flash memory, a hard disk drive, and an optical disk drive.

[0077] The memory 313 stores an instruction set (eg, software) for operating the electronic device 310 . The instruction set for operating the electronic device 310 is executed by the processor 312 .

[0078] According to one embodiment, at least one external controller and the electronic device 310 may be paired via a wireless communication channel. For example, the wireless communication channel may be a wireless communication channel based on Bluetooth (BLE), LoRa, EnOcean, radio frequency (RF) communication, Zigbee, or infrared communication.

[0079] According to an embodiment, the channel for pairing the electronic device 310 with the at least one external controller and the channel for pairing and connecting the electronic device 310 with the aerosol generating device 320 may be different channels (or paths). For example, the electronic device 310 and the at least one external controller may be paired via a Bluetooth communication channel, whereas, as described below, the electronic device 310 and the aerosol generating device 320 may be connected to the USB port 315 of the electronic device 310 via the adapter 330.

[0080] According to an embodiment, the electronic device 310 and the aerosol generating device 320 may be connected via an adapter 330. For example, the adapter 330 may be connected to the USB port 315 of the electronic device 310. The communication driver and the virtual smoking program stored in the adapter 330 may be installed from the adapter 330 to the electronic device 310.

[0081] According to an embodiment, when the adapter 330 is connected (or inserted into, contacted) to the USB port 315 of the electronic device 310, the communication driver and the virtual smoking program may be automatically installed to the electronic device 310. For example, the communication driver and the virtual smoking program may be installed or stored in the memory 313.

[0082] According to one embodiment, after the adapter 330 is connected to the USB port 315 of the electronic device 310 , the communication driver and the virtual smoking program may be installed upon receiving additional input from the user.

[0083] According to an embodiment, the processor 312 may execute a virtual smoking program. For example, the virtual smoking program may be automatically run after the virtual smoking program is installed. For another example, the virtual smoking program may be run when a start smoking signal is received from the user or a start smoking signal is received from the aerosol generating device 320. The processor 312 may generate a virtual smoke image based on the sensing information about the state of the aerosol generating device 320 received from the aerosol generating device 320 through the adapter 330.

[0084] A virtual reality, augmented reality, or metaverse image that generates smoke can be provided to the user through the display 314 of the electronic device 310. In one embodiment, the display 314 can be controlled to be transparent or opaque according to the purpose of use. When the electronic device 310 provides a virtual reality or metaverse image, the display 314 can be controlled to be opaque, and when the electronic device 310 provides an augmented reality image, the display 314 can be controlled to be transparent.

[0085] For example, the display 314 may be made of a plastic or glass material like polycarbonate, but is not limited thereto. In addition, a coating method of coating at least one of a light reflecting and anti-glare coating, an anti-fog coating, and an ultraviolet blocking coating may be applied in the display 314 .

[0086] According to an example, the system 300 may further include a product. The electronic device 310 may receive product content information from the product. The electronic device 310 may change the displayed image on the display 314 based on the product content information.

[0087] In one embodiment, the display 314 may display an image representing tobacco smoke, and the electronic device 310 may change at least any one of the color, amount and shape of the tobacco smoke displayed on the display 314 based on the product content information.

[0088] The electronic device 310 may further include a camera (not shown). When the user uses the electronic device 310, the camera may help the user use the electronic device 310 by scanning the real world environment.

[0089] The electronic device 310 may be installed with an application program to provide different experiences to the user. For example, in order to provide the user with a smoking experience close to reality, a virtual space smoking system may be provided by installing a smoking application. However, the present disclosure is not limited thereto.

[0090] The electronic device 310 may also include a sensor for sensing or predicting the smoking behavior of the user of the system 300. For example, the sensor may include a surface electromyography sensor or a brain wave measurement module attached to the user's skin to sense the user's smoking behavior. The electronic device 310 may include a communication device for communicating with the article. However, the present disclosure is not limited thereto.

[0091] Figure 4 is a flow chart of a method for outputting a virtual smoke image according to an embodiment.

[0092] The following steps 410 to 460 can be referred to above. Figure 3 The electronic device 310 is described as executing.

[0093] The following steps 410 to 460 may be executed when the electronic device 310 is executing an existing application program, that is, the existing application program and the virtual smoking program may be executed at the same time.

[0094] In step 410, the electronic device 310 may communicate with at least one external controller (eg, Figure 1 A connection request is received from the aerosol generating device 320 in a paired state with an external controller 150, 160).

[0095] According to one embodiment, at least one external controller and the electronic device 310 may be paired via a wireless communication channel. For example, the wireless communication channel may be a wireless communication channel based on Bluetooth Low Energy (BLE), LoRa, EnOcean, radio frequency (RF) communication, Zigbee, or infrared communication.

[0096] In one embodiment, when the electronic device 310 receives the connection request of the aerosol generating device 320 , the electronic device 310 may execute an existing application, thereby executing the existing application (eg, a game) using at least one external controller.

[0097] In one embodiment, the electronic device 310 may receive a connection request from the aerosol generating device 320 through the adapter 330. Specifically, the connection request may be received from the aerosol generating device 320 that has established a wireless communication channel with the adapter 330 through the adapter 330 connected to the electronic device 310. For example, the wireless communication channel may be a wireless communication channel based on BLE, LoRa, EnOcean, RF communication, Zigbee, or infrared communication.

[0098] In one embodiment, the electronic device 310 may receive a connection request from the aerosol generating device 320 via a wireless communication channel. For example, the connection request may be received from the aerosol generating device 320 that has directly established a wireless communication channel with the electronic device 310. The wireless communication channel may be at least one of short-range communication and cellular communication. For another example, the aerosol generating device 320 and the electronic device 310 may be connected via a USB cable. In the above embodiment, the virtual smoking program may be pre-stored in the electronic device 310.

[0099] In one embodiment, the electronic device 310 may directly receive a connection request between the aerosol generating device 320 and the electronic device 310 from the user through a user interface (UI) of the electronic device 310 .

[0100] In step 420, the electronic device 310 may determine a controller type of the aerosol generating device 320. For example, the controller type may include a left-hand controller type, a right-hand controller type, or a mouth-held controller type.

[0101] In one embodiment, the user may directly input the controller type of the aerosol generating device 320 through a user interface (UI) of the electronic device 310 .

[0102] In an embodiment, the controller type of the aerosol generating device 320 may correspond to the controller type of an external controller that does not sense movement at the time the connection request is received from the aerosol generating device 320 .

[0103] In one embodiment, the controller type of the aerosol generating device 320 may be determined based on sensing information of a gyroscope sensor or a position sensor of the aerosol generating device 320. Alternatively, the controller type of the aerosol generating device 320 may be determined based on sensing information of a pressure sensor or a capacitive sensor of the aerosol generating device 320. For example, if the sensing information of the pressure sensor of the aerosol generating device 320 indicates that an object (e.g., the lips of a user) contacts one side of the aerosol generating device 320, the controller type of the aerosol generating device 320 may be determined to be a mouth-held controller type.

[0104] In step 430, when the determined controller type corresponds to the target external controller type in the at least one external controller, the electronic device 310 stops pairing with the target external controller and connects to the aerosol generating device 320.

[0105] In one embodiment, the controller type of the at least one external controller may be preset. Alternatively, the controller type may be determined when the electronic device 310 is paired with the at least one external controller. Alternatively, the controller type may be determined using information sensed by a sensor of the at least one external controller.

[0106] In one embodiment, the controller type of the aerosol generating device 320 is determined to be a right-hand controller type, which corresponds to the target external controller type (i.e., right-hand controller) held by the user's right hand among the at least one external controller, and the electronic device 310 may stop pairing with the target external controller and connect with the aerosol generating device 320. The user may control an existing application (e.g., play a game) executed by the electronic device 310 through the aerosol generating device 320 on the right hand and the external controller on the left hand.

[0107] In step 440, the electronic device 310 may receive state sensing information of the aerosol generating device 320 from the aerosol generating device 320 connected to the electronic device 310. For example, the sensing information may include distance information between the aerosol generating device 320 and an object (e.g., a part of the user's body), sensing value information of a capacitive sensor of the aerosol generating device 320, position change information of the aerosol generating device 320, camera monitoring information of the aerosol generating device 320, smoking patterns learned using a pressure sensor and a button of the aerosol generating device 320, camera data and pressure sensor data, and at least one of button input information. Figure 5 The sensing information of the aerosol generating device 320 is described in more detail.

[0108] In one embodiment, the sensing information of the aerosol generating device 320 may include sensing information for virtual smoking and sensing information for controlling an existing application program run by the electronic device 310. For example, the sensing information for virtual smoking may include sensing information for outputting a virtual smoke image. For example, the sensing information for controlling the electronic device 310 may include at least one of position change information of the aerosol generating device 320 and button input information. The position change information of the aerosol generating device 320 may be sensing information of a gyroscope sensor, an acceleration sensor, or a three-degree-of-freedom / six-degree-of-freedom (3dof / 6dof) sensor, etc.

[0109] In one embodiment, when the user is controlling an existing application program running on the electronic device 310, the electronic device 310 may receive position change information that the aerosol generating device 320 moves upward beyond a certain threshold (meaning that the user brings the aerosol generating device 320 close to the lips) and / or button input information of the aerosol generating device 320 (meaning that the user starts smoking). For example, when the smoking function of the aerosol generating device 320 is activated, the control function of the existing application program through the aerosol generating device 320 may be temporarily stopped, and a virtual smoke image may be generated and output through the following steps 450 and 460. Even if the control of the existing application program using the aerosol generating device 320 is temporarily stopped, the functions of other external controllers will not be interrupted.

[0110] In one embodiment, the sensing information for controlling the electronic device 310 may also include at least a portion of sensing information for virtual smoking. For example, when the user is controlling an existing application program run by the electronic device 310, the electronic device 310 may receive position change information of the aerosol generating device 320 moving upward beyond a certain threshold (meaning that the user has brought the aerosol generating device 320 close to the lips) and capacitance change information (meaning that the aerosol generating device 320 has contacted the user's lips). For another example, when the user is controlling an existing application program run by the electronic device 310, the electronic device 310 may receive button input information of the aerosol generating device 320 (meaning that the user has started smoking) and pressure sensor data (meaning that the user has started inhaling the aerosol generating device 320). When the sensing information for virtual smoking is received from the aerosol generating device 320, the electronic device 310 temporarily stops the control function through the aerosol generating device 320, and generates and outputs a virtual smoke image through the following steps 450 and 460.

[0111] In one embodiment, after the user starts to inhale the aerosol generating device 320 , the aerosol generating device 320 may be used again to control the electronic device 310 in each interval of repeated inhalation of the aerosol generating device 320 .

[0112] In one embodiment, the control of the electronic device 310 by the aerosol generating device 320 may be implemented based on the position change information of the aerosol generating device 320 and the camera monitoring information of the aerosol generating device 320 .

[0113] In step 450 , the electronic device 310 may generate a virtual smoke image based on the sensing information for virtual smoking.

[0114] In one embodiment, the electronic device 310 may determine the properties of the virtual smoke based on the sensing information. For example, the electronic device 310 may determine the properties of the virtual smoke based on the sensing information for virtual smoking described in step 440. For example, the properties of the virtual smoke may include at least one of the size, shape, duration, volume, concentration, and color of the virtual smoke. This specification does not limit this. The electronic device 310 may generate a virtual smoke image based on the properties of the virtual smoke.

[0115] In step 460 , the virtual smoke image may be outputted via the display of the electronic device 310 .

[0116] In one embodiment, the electronic device 310 can determine the exhalation moment based on a first moment and a second moment sensed by the aerosol generating device 320, wherein the first moment is the moment when the distance between one side of the aerosol generating device 320 and a part of the user's body reaches within a first threshold; the second moment is the moment after the first moment when the distance between one side of the aerosol generating device 320 and a part of the user's body exceeds a second threshold.

[0117] In one embodiment, the electronic device 310 can determine the exhalation time based on a first time and a second time sensed by the aerosol generating device 320, wherein the first time is the time when the capacitance value of the aerosol generating device 320 reaches a first threshold; the second time is the second time after the first time when the capacitance value of the aerosol generating device 320 exceeds the second threshold.

[0118] In one embodiment, the electronic device 310 may superimpose the virtual smoke image on a reference image (e.g., a game image) of an existing application program executed by the electronic device 310. For example, the electronic device 310 may superimpose the image by using a layer of output target image content and a virtual smoke image layer. The user may observe the superimposed image.

[0119] In one embodiment, the virtual smoke image may be output in a manner of being synthesized in a scene of the reference image content. For example, the virtual smoke may be output in a manner of being displayed in a scene of the reference image.

[0120] Even if the smoke cannot be visually observed in reality, the user can observe the smoke through the virtual smoke image and satisfy the smoking sensation through the observed smoke.

[0121] Figure 5 An example of an aerosol generating device according to an embodiment is shown.

[0122] Reference Figure 5 , aerosol generating device 500 (e.g., Figure 1 an aerosol generating device 120, or Figure 3 The aerosol generating device 500 may be in the shape of a cigarette. For example, the length and diameter of the aerosol generating device 500 may be substantially the same as those of a conventional cigarette, but is not limited thereto. The aerosol generating device 500 may be divided into a first end portion 510, a middle portion 520, and a second end portion 530.

[0123] The first end 510 may be a portion that reproduces the ignition portion of a cigarette. The first end 510 may include an LED. For example, the first end 510 may include a red LED.

[0124] The second end 530 may contain nicotine. In one embodiment, the surface of the second end 530 may be coated with nicotine, or the second end 530 may contain an aerosol generating product. For example, when the aerosol generating product is heated, an aerosol containing nicotine may be generated. When a user holds the second end 530 of the aerosol generating device 500 and inhales the aerosol, nicotine may be provided to the user. In addition to the aerosol generating product, the second end 530 may also include a flavoring portion containing a flavoring.

[0125] In one embodiment, the middle portion 520 is used to connect the first end 510 and the second end 530. The total length of the aerosol generating device 500 is determined by the length of the middle portion 520. When manufacturing the aerosol generating device 500, the length of the middle portion 520 may correspond to the length of a conventional cigarette.

[0126] In another embodiment, the aerosol generating device 500 is an electronic device that includes other electronic devices or circuits inside (e.g., the middle part 520). For example, the aerosol generating device 500 may include a sensor for obtaining sensing information for sensing the state of the aerosol generating device 500, or sensing the smoking behavior of the user. The sensors may include sensors for detecting distance changes, sensors for detecting position changes, sensors for detecting capacitance changes, and pressure sensors, etc. The outside of the housing of the aerosol generating device 500 may include other electronic devices as electronic devices. For example, the aerosol generating device 500 may include a camera for observing the smoking behavior of the user. The aerosol generating device 500 may have internal or external buttons that implement power control functions, smoking pattern learning functions, smoking detection functions, or electronic device 310 control functions. The aerosol generating device 500 may include a sensor that is connected to an XR device (e.g., Figure 1 The electronic device 110, or Figure 3 The electronic device 310) is a communication device for communicating. However, the aerosol generating device of the present disclosure is not limited to including sensors, cameras or buttons. The method of determining the exhalation time using different sensing information will be described below. The same or similar content will be omitted.

[0127] <Distance information of distance detection sensor>

[0128] The distance detection sensor according to an embodiment may be located inside or outside the second end 530. The distance detection sensor according to an embodiment may be an infrared sensor, a laser radar sensor, an ultrasonic sensor, etc. However, the distance detection sensor of the present disclosure is not limited thereto. A method for determining the exhalation moment when a user starts to exhale using a distance detection sensor according to an embodiment will be described in detail below.

[0129] According to one embodiment, when the user starts to smoke, the distance detection sensor can sense the distance between the object (e.g., the user's lips) and the aerosol generating device 500. The aerosol generating device 500 can be connected to the aerosol generating device 500 through an adapter (e.g., Figure 1 adapter 130, or Figure 3 The adapter 330 of the aerosol generating device 500 transmits the sensing information of the distance detection sensor to the electronic device 110 (or the electronic device 310). The electronic device 310 may determine the exhalation timing based on the sensing information of the distance detection sensor received from the aerosol generating device 500.

[0130] According to an embodiment, the smoking start step includes the step of the user putting the aerosol generating device 500 into the mouth and inhaling. The user can turn on the power to start heating the aerosol generating product in the aerosol generating device 500. After the aerosol generating device 500 is turned on, the heater for heating the aerosol generating product can be preheated.

[0131] According to an embodiment, the step of sensing the distance between an object (e.g., a part of a user's body) and the aerosol generating device 500 by the distance detection sensor may include the step of detecting a first moment when the object contacts the second end 530 of the aerosol generating device 500; and the step of detecting a second moment when the object moves away from the second end 530 of the article. For example, the step of detecting the first moment may include the step of detecting that the distance between the second end 530 of the aerosol generating device 500 and the object is close to a first threshold value by the distance detection sensor. For example, the step of detecting the second moment may include the step of detecting that the distance between the object and the second end 530 of the aerosol generating device 500 after contacting the second end 530 is greater than a second threshold value.

[0132] According to one embodiment, the step of determining the exhalation moment when the user starts to exhale may include the step of determining the exhalation moment based on the interval between the first moment and the second moment. The first moment may be the moment when the user starts to inhale smoke. The interval between the first moment and the second moment is the time when the user inhales smoke, and the first breathing time of the user may be predicted based on the smoke inhalation time. The second moment may be the moment when the user starts to inhale. The second breathing moment may be predicted based on the interval between the first moment and the second moment and the first breathing time. The second breathing moment may be the moment when the user starts to exhale. For example, the second breathing moment may be determined as the exhalation moment.

[0133] According to one embodiment, since the smoke inhalation time of the user is the interval between the first moment and the second moment, the first breathing time as the user's inhalation time can be estimated to be similar to the smoke inhalation time. Thereafter, the electronic device 310 can determine the moment when the user starts to exhale, that is, the second breathing moment, as the moment after the first breathing time from the second moment. For example, assuming that the time from the first moment when the object contacts the second end 530 to the second moment when the object leaves the second end 530 is 1 second, it can be predicted that the first breathing time is 1 second, and the user's inhalation time is similar to the first breathing time, which is also 1 second. Thus, the second breathing moment after the first breathing time from the second moment can be predicted as the moment after 1 second after the object is separated from the aerosol generating device 500. Therefore, the electronic device 310 can determine the exhalation moment as the moment after 1 second since the object is separated.

[0134] According to an embodiment, the step of determining the exhalation time based on the distance sensing information may include the step of estimating the inhalation volume and the breathing volume of the user based on the interval between the first moment and the second moment. The inhalation volume of the user may include: the inhalation volume of the aerosol inhaled by the user through the aerosol generating device 500 when the aerosol generating device 500 is in contact with the object. The breathing volume of the user may include: the inhalation volume inhaled by the user when the aerosol generating device 500 is not in contact with the object. Based on the estimated breathing volume and inhalation volume, the exhalation time may be determined.

[0135] According to one embodiment, the step of determining the exhalation moment based on the sensing information may include the step of determining whether the aerosol generating device 500 satisfies a preset operating condition; and the step of determining the exhalation moment based on the determination that the aerosol generating device 500 satisfies the operating condition. The preset operating conditions include whether the user turns on / off the power of the aerosol generating device 500; and whether at least one of the suction sensor in the aerosol generating device and the pressure sensor in the aerosol generating device is operating. Objects other than the designated objects (such as fingers, etc.) may change the sensing value of the distance detection sensor. Therefore, the distance detection sensor may be operated only when the user turns on / off the power, or the distance detection sensor may be activated by sensing the user's inhalation through a suction sensor or a pressure sensor, etc. However, the inhalation sensor of the present disclosure is not limited to a suction sensor or a pressure sensor.

[0136] According to one embodiment, the step of sending the sensing information to the electronic device 310 through the adapter 330 includes the step of sending virtual smoke type information; and the step of sending information about the delay generated in the communication between the aerosol generating device 500 and the electronic device 310. The type of virtual smoke can be changed according to the user's choice, or the aerosol generating product or other flavoring part built into the second end 530. The sensing information can also include information such as the user's smoking amount and breathing amount, but the present disclosure is not limited to this. When the exhalation moment does not match the user's actual smoking behavior, or the accurate predicted moment cannot be obtained, additional work can be performed to perform "exhalation prediction sensing" separately.

[0137] <Capacitive sensor sensing value information>

[0138] According to one embodiment, the capacitive sensor may be located inside or outside the second end 530. According to one embodiment, the capacitive sensor may be a sensor based on a capacitor configuration. However, the capacitive sensor of the present disclosure is not limited to the above method. Hereinafter, a method for determining the exhalation moment when the user starts to exhale using a capacitive sensor according to one embodiment will be described in detail.

[0139] According to an embodiment, when the user starts smoking, the capacitive sensor may monitor the capacitance change of the capacitive sensor when an object (e.g., the user's lips) contacts the aerosol generating device 500. For example, the capacitive sensor may sense the capacitance change value. The aerosol generating device 500 may send the sensing information of the capacitive sensor to the electronic device 310 through the adapter 330. The electronic device 310 may determine the exhalation moment based on the sensing information of the capacitive sensor received from the aerosol generating device 500.

[0140] According to one embodiment, the smoking start step includes the user touching the aerosol generating device 500 to the lips and inhaling. The user can turn on the power to start heating the aerosol generating product in the aerosol generating device 500. After the aerosol generating device 500 is turned on, the heater for heating the aerosol generating product can be preheated.

[0141] According to an embodiment, the step of sensing the change in capacitance when the object contacts the aerosol generating device 500 by the capacitive sensor may include: a step of detecting a first moment when the object contacts the second end 530 of the aerosol generating device 500; and a step of detecting a second moment when the object moves away from (leaves) the second end 530 of the article. For example, the step of detecting the first moment may include: the capacitance sensor detecting that the capacitance sensed when the second end 530 of the aerosol generating device 500 contacts the object is within a first threshold. For example, the step of detecting the second moment may include: the capacitance sensor detecting that the capacitance sensed when the object leaves the second end 530 of the aerosol generating device 500 after contacting the aerosol generating device 500 exceeds a second threshold.

[0142] According to one embodiment, the step of the electronic device 310 predicting the exhalation moment based on the capacitive sensing information may include the step of predicting the exhalation moment based on the interval between the first moment and the second moment. The first moment may be the moment when the user starts to inhale smoke. The interval between the first moment and the second moment is the time when the user inhales the smoke, and the first breathing time of the user can be predicted based on the smoke inhalation time. The second moment may be the moment when the user starts to inhale. The second breathing moment can be predicted based on the interval between the first moment and the second moment and the first breathing time. The second breathing moment may be the moment when the user starts to exhale. For example, the second breathing moment can be determined as the exhalation moment.

[0143] According to one embodiment, since the smoke inhalation time of the user is the interval between the first moment and the second moment, the first breathing time as the user's inhalation time can be estimated to be similar to the smoke inhalation time. Thereafter, the electronic device 600 can determine the moment when the user starts to exhale, that is, the second breathing moment, as the moment after the first breathing time from the second moment. For example, assuming that it is 1 second from the first moment when the object contacts the second end 530 to the second moment when the object leaves the second end 530, the first breathing time can be estimated to be 1 second, and the user's inhalation time is predicted to be similar to the first breathing time, also 1 second. Thus, the second breathing moment after the first breathing time from the second moment can be predicted as the moment after 1 second after the object is separated from the aerosol generating device 500. Therefore, the electronic device 310 can determine the exhalation moment as the moment after 1 second since the object is separated.

[0144] According to an embodiment, the step of determining the exhalation time based on the capacitance sensing information may include the step of estimating the inhalation amount and the breathing amount of the user based on the interval between the first moment and the second moment. The inhalation amount of the user may include the inhalation amount of the aerosol inhaled by the user through the aerosol generating device 500 when the aerosol generating device 500 is in contact with the object. The breathing amount of the user may include the inhalation amount inhaled by the user when the aerosol generating device 500 is not in contact with the object. Based on the estimated breathing amount and inhalation amount, the exhalation time may be determined.

[0145] According to one embodiment, the step of the electronic device 310 determining the exhalation moment based on the capacitive sensing information may include: a step of judging whether the aerosol generating device 500 satisfies a preset operating condition; and a step of determining the exhalation moment based on the judgment that the aerosol generating device 500 satisfies the operating condition. The preset operating condition includes whether the user turns on / off the power of the aerosol generating device 500; and whether at least one of the suction sensor in the aerosol generating device and the pressure sensor in the aerosol generating device is operating. Objects other than the designated objects (such as fingers, etc.) may change the sensing value of the capacitive sensor. Therefore, the capacitive sensor may be operated only when the user turns on / off the power, or the capacitive sensor may be activated by sensing the user's inhalation through a suction sensor or a pressure sensor, etc. However, the inhalation sensor of the present disclosure is not limited to a suction sensor or a pressure sensor.

[0146] According to an embodiment, the step of sending the sensing information of the capacitive sensor to the electronic device 310 through the adapter 330 includes: sending virtual smoke type information; and sending communication delay information between the aerosol generating device 500 and the electronic device 310. The type of virtual smoke can be changed according to the user's selection, or the aerosol generating product or other flavoring part built into the second end 530. The sensing information may also include information such as the user's smoking amount and breathing amount, but the present disclosure is not limited thereto.

[0147] <Location change information>

[0148] According to one embodiment, the position change sensor may be located inside or outside the second end 530. The position change sensor according to one embodiment may be a gyroscope sensor, an acceleration sensor, or a three-degree-of-freedom / six-degree-of-freedom (3dof / 6dof) sensor, etc. However, the position change sensor of the present disclosure is not limited thereto. Hereinafter, a method for determining the exhalation moment when the user starts to exhale using a position change sensor according to one embodiment will be described in detail.

[0149] According to an embodiment, when the user starts to smoke, the position change sensor may sense the position change between the object (e.g., the user's lips) and the aerosol generating device 500. The aerosol generating device 500 may transmit the sensing information of the position change sensor to the electronic device 310 through the adapter 330. The electronic device 310 may determine the exhalation timing based on the sensing information of the position change sensor.

[0150] According to an embodiment, the step 760 of sensing the position change between the object and the aerosol generating device 500 by the position change sensor may include: a step of detecting a first moment when the object contacts the second end 530 of the aerosol generating device 500; and a step of detecting a second moment when the object moves away from the second end 530 of the article. For example, the step of detecting the first moment may include: detecting, by the position change sensor, that the positions of the second end 530 of the aerosol generating device 500 and the object are close to a first threshold, and detecting a change in acceleration when the user moves the aerosol generating device 500 to the object. For example, the step of detecting the second moment may include: detecting that after the second end 530 of the aerosol generating device 500 contacts the object, the position at which the object moves away exceeds a second threshold, and sensing a change in acceleration when the user takes the aerosol generating device 500 away from the object.

[0151] According to one embodiment, the position change can be measured by comparing the x, y, z-axis orientation of the aerosol generating device 500 when the user puts down the hand with the x, y, z-axis orientation of the aerosol generating device 500 when the user brings the aerosol generating device 500 into contact with the object. When the user moves the hand to move the aerosol generating device 500, the measured acceleration will increase first and then decrease until it stops. The position change can be predicted by measuring the change interval of the acceleration, and the user's smoking time can be estimated based on the predicted position change.

[0152] According to an embodiment, the step of predicting the exhalation time based on the position change sensing information by the electronic device 310 may include the step of determining the exhalation time based on the interval between the first time and the second time. Below, the process of determining the exhalation time is the same as the process of using the capacitive sensor, so repeated description is omitted.

[0153] <Camera monitoring information>

[0154] According to an embodiment, the camera may be located on an outer surface of the housing of the aerosol generating device 500. Hereinafter, a method of determining an exhalation moment at which a user starts to exhale using a camera according to an embodiment will be described.

[0155] When a user according to an embodiment starts smoking, the camera can monitor the movement of an object (e.g., the user's lips). The aerosol generating device 500 can send the monitoring information (i.e., sensing information) of the camera to the electronic device 310 through the adapter 330. The electronic device 310 can determine the exhalation moment based on the monitoring information of the camera.

[0156] According to an embodiment, the step of monitoring the distance between the object and the aerosol generating device 500 by the camera may include: a step of detecting a first moment when the object changes its action in order to contact the aerosol generating device 500 at the second end 530 of the aerosol generating device 500; and a step of detecting a second moment when the object changes its action in order to separate the second end 530 of the aerosol generating device 500 from the aerosol generating device 500. For example, the step of detecting the first moment may include the camera detecting a first action of bringing the second end 530 of the aerosol generating device 500 to the lips to contact the object. For example, the step of detecting the second moment may include the camera detecting a second action of moving the second end 530 of the aerosol generating device 500 away from the object.

[0157] According to an embodiment, the step of determining the exhalation time based on the monitoring information of the camera by the electronic device 310 includes the step of determining the exhalation time based on the interval between the first time and the second time. Below, the process of determining the exhalation time is the same as the process of using the capacitive sensor, so repeated description is omitted.

[0158] <Smoking mode learned using pressure sensor and button>

[0159] According to one embodiment, a pressure sensor and a button (not shown) may be provided in the aerosol generating device 500. The pressure sensor may be located inside or outside the second end 530. The pressure sensor senses whether the user inhales by measuring the pressure generated when the user places the second end on the lips or holds the second end to smoke. The button may have the function of turning on / off the power of the aerosol generating device 500, and may include an input function for storing the user's smoking mode. The button may include a function of determining whether to start the power or store the smoking mode according to the user input. For example, a long press has the function of turning on / off the power, a short press has the function of storing the smoking mode, or it may also be a function of sending sensing information such as changing the virtual smoke type. However, the buttons of the present disclosure are not limited to the above functions.

[0160] According to one embodiment, when the user starts smoking, the pressure sensor and the button can be used to make the aerosol generating device 500 learn the smoking pattern. The electronic device 310 can determine the exhalation time based on the learned smoking pattern, and determine the volume and concentration of the virtual smoke. The aerosol generating device 500 can send the smoking pattern (i.e., sensing information) to the electronic device 310 through the adapter 330.

[0161] According to one embodiment, the step of learning the smoking mode using the pressure sensor and the button may include the step of detecting a first moment, wherein the first moment is the moment when the object contacts the second end 530 of the aerosol generating device 500 so that the pressure sensing value measured by the pressure sensor exceeds a first threshold; and the step of detecting a second moment, wherein the second moment is the moment when the object separates from the second end 530 after the first moment so that the pressure sensing value of the pressure sensor is within the second threshold; and the step of detecting a third moment, wherein the third moment is the moment when the button of the aerosol generating device 500 is operated after the second moment.

[0162] According to one embodiment, in the learning method of the smoking pattern, when the user starts to inhale to smoke, the moment of pressure change can be measured and saved by a pressure sensor. Thereafter, the moment when the sensing value of the pressure sensor is restored when the user ends the inhalation is stored. The time difference (time duration) between the start and end of the inhalation is the user's smoke inhalation time, which is similar to the user's inhalation time. Then, when the user exhales the inhaled smoke, the button input of the aerosol generating device 500 is pressed to start exhalation. When the user ends the exhalation, the user stops the button input. The time difference between the start and end of the exhalation is similar to the user's exhalation time. The aerosol generating device 500 can execute the above-mentioned user smoking pattern at least once, and generate a user smoking pattern algorithm based on this, and store the smoking pattern learned based on the algorithm.

[0163] According to one embodiment, the step of storing the smoking pattern may include the step of storing the smoking pattern based on the interval between the first moment and the second moment and the interval between the second moment and the third moment; and the step of collecting at least one data about the interval between the first moment and the second moment and the interval between the second moment and the third moment.

[0164] According to an embodiment, the step of determining the exhalation time based on the learned smoking pattern by the electronic device 310 may include the step of determining the exhalation time based on the interval between the first time and the second time. The process of determining the exhalation time is the same as the process of using the capacitive sensor, so the repetitive content is omitted.

[0165] According to one embodiment, the step of determining the volume and concentration of virtual smoke based on the learned smoking pattern may include the steps of storing comparison data of the operating time of the pressure sensor and the operating time of the button; and storing the volume and concentration of the virtual smoke determined based on the comparison data into the learned smoking pattern.

[0166] According to one embodiment, the volume and concentration of the virtual smoke may be different according to the user's smoking pattern. For example, if the user's exhalation time is shorter than the user's suction time, the volume of the virtual smoke may be increased to produce thicker smoke. If the pressure sensor detects that the user's suction time is 2 seconds, and the time it takes for the user to press and release the button is 1.5 seconds, the volume and concentration of the virtual smoke may be adjusted to exhale more and thicker smoke. That is, since the user's suction time is the time to inhale the smoke, and the user's button operation time is the exhalation time, if the user's exhalation time is less than the user's suction time, it means that the user exhales the same amount of smoke in a shorter time, and therefore, the electronic device 310 needs to achieve more and thicker smoke in a shorter time.

[0167] <Camera data and pressure sensor data>

[0168] According to an embodiment, a pressure sensor may be provided in the aerosol generating device 500. The pressure sensor may be located inside or outside the second end portion 530. The pressure sensor may measure the pressure generated by the lips of the user to sense whether inhalation is performed.

[0169] Also refer to Figure 3 and Figure 5 According to one embodiment, when the user starts smoking, the electronic device 310 receives camera data sensing the user's hand movement from the camera. When the user holds the second end 530 in the mouth to smoke, the pressure sensor can obtain pressure sensor data. The electronic device 310 can predict the exhalation time based on the camera data and the pressure sensor data received through the adapter, that is, the sensing information.

[0170] According to one embodiment, the electronic device 310 may be installed with an application program that monitors the user's hand movements, thereby sensing the user's smoking movement data through a camera that monitors the user's hand movements. The camera may monitor the situation in which the user brings the aerosol generating device 500 to the mouth to inhale the smoke, inhales the smoke into the lungs through inhalation, and then takes the aerosol generating device 500 away from the mouth to exhale the smoke. The data received from the camera and the application program installed in the electronic device 310 may analyze the user's hand movements as described above to determine the amount of inhalation and the time of exhalation, etc.

[0171] According to one embodiment, the operation of the electronic device 310, the camera, and the aerosol generating device 500 is as follows. The electronic device 310 with the smoking device application installed is connected to the aerosol generating device 500 through the adapter 330. When the user brings the aerosol generating device 500 to the mouth to smoke, the application of the electronic device 310 can use the image captured by the camera (e.g., camera data) to analyze the user's hand movement. When it is determined through the image of the camera that the aerosol generating device 500 has reached the mouth and the pressure sensor is working (e.g., the measured pressure value is above the threshold), it can be determined that the user has started smoking. For example, the aerosol generating device 500 can send the pressure value measured by the pressure sensor to the electronic device 310 through the adapter 330. After the user starts smoking, the image of the camera can be used to monitor the user's action of inhaling smoke into the lungs through inhalation, that is, the action of separating the aerosol generating device 500 from the mouth. By analyzing the interval between the start time of smoking and the start time of inhalation, the inhalation amount and the inhalation amount can be estimated, and the exhalation time can be determined based on the estimated inhalation amount and the inhalation amount.

[0172] According to an embodiment, the step of determining the exhalation time based on the above monitoring result, i.e., the sensing information, may include the step of determining the exhalation time based on the interval between the first time and the second time. Below, the process of determining the exhalation time is the same as the process of using the capacitive sensor, so repeated description is omitted.

[0173] Figure 6 is a flow chart of a method for outputting a virtual smoke image according to an embodiment.

[0174] According to one embodiment, the above reference Figure 4 The described step 450 may include the following steps 610 and 620 .

[0175] In step 610, the electronic device 310 may generate a signal based on the aerosol generated by the aerosol generating device 320 (or Figure 1 an aerosol generating device 120, or Figure 5 The aerosol generating device 500) receives the sensing information to determine the properties of the virtual smoke. For example, the properties of the virtual smoke may include at least one of the size, shape, duration, volume, concentration, and color of the virtual smoke. This specification does not limit this.

[0176] In one embodiment, the size, duration, volume and concentration of the virtual smoke are proportional to the predicted smoke inhalation time or the inhalation time based on the smoke inhalation time, wherein the smoke inhalation time is predicted using at least one of the distance information between the aerosol generating device 320 and the object (e.g., a part of the user's body), the sensing value information of the capacitive sensor of the aerosol generating device 120, the position change information of the aerosol generating device 320, the camera monitoring information of the aerosol generating device 120, the smoking pattern learned using the pressure sensor and button of the aerosol generating device 320, the camera data and the pressure sensor data, and the button input information. For example, when the inhalation time is longer, the duration of the virtual smoke can be extended by the same or the same proportion.

[0177] In one embodiment, the shape of the virtual smoke can be determined based on the monitoring information of the camera. When the camera of the electronic device 310 receives object information, for example, information on monitoring the lip movements of the user, the monitoring information of the camera may include information related to the shape of the lips. For example, if the user's lips are slightly pursed, a thin and long virtual smoke shape may be determined.

[0178] In one embodiment, the volume, concentration and color of the virtual smoke can be determined according to the user's smoking pattern. For example, when the user's exhalation time is shorter than the user's smoke inhalation time, the volume of the virtual smoke can be increased to generate thicker smoke, and the color of the virtual smoke can be determined to be a darker color.

[0179] In one embodiment, the electronic device 310 may directly receive user input related to the properties of the virtual smoke. For example, the virtual smoke may be a general achromatic color, or may be displayed in multiple colors such as blue and purple according to the user input. For another example, the shape of the virtual smoke may be a donut shape.

[0180] In one embodiment, the state sensing information of the aerosol generating device 320 is pressure sensor data, and may include airflow sensing information based on the user's inhalation. If the internal flow pressure of the aerosol generating device 320 to which the pressure sensor is attached is high, even if the inhalation time is short, a large and persistent virtual smoke may be determined.

[0181] In step 620 , the electronic device 310 may generate virtual smoke information based on the properties of the virtual smoke. In one embodiment, the virtual smoke information may be generated by adjusting the preset basic properties based on the properties of the virtual smoke determined in step 610 .

[0182] Figure 7 is a flow chart of a method for outputting a virtual smoke image according to an embodiment.

[0183] According to one embodiment, the above reference Figure 4The illustrated step 460 may include the following steps 710 and 720 .

[0184] In step 710, the electronic device 310 determines an exhalation moment at which the user of the aerosol generating device 320 starts to exhale based on the sensing information about the state of the aerosol generating device 320. The sensing information about the state of the aerosol generating device 320 includes at least one of distance information between the aerosol generating device 320 and an object (e.g., a part of the user's body), a sensing value of a capacitive sensor of the aerosol generating device 320, position change information of the aerosol generating device 320, camera monitoring information of the aerosol generating device 320, a smoking pattern learned using a pressure sensor and a button of the aerosol generating device 320, camera data and pressure sensor data, and button input information.

[0185] In step 720 , the electronic device 310 may output a virtual smoke image at the exhalation moment. In one embodiment, the virtual smoke image is output via a display of the electronic device 310 .

[0186] In one embodiment, the electronic device 310 may superimpose the virtual smoke image on a reference image (e.g., a game image) of an existing application program executed by the electronic device 310. For example, the electronic device 310 may superimpose the image by using a layer of output target image content and a virtual smoke image layer. The user may observe the superimposed image.

[0187] In one embodiment, the virtual smoke image may be output in a manner of being synthesized in a scene of the reference image content. For example, the virtual smoke may be output in a manner of being displayed in a scene of the reference image.

[0188] Even if the smoke cannot be visually observed in reality, the user can observe the smoke through the virtual smoke image and satisfy the smoking sensation through the observed smoke.

[0189] The above-described device can be implemented by hardware components, software components, and / or a combination of hardware components and software components. For example, the device and components described in the embodiment can be embodied using one or more general-purpose computers or special-purpose computers such as processors, controllers, arithmetic logic units (arithmetic logic units, ALU), digital signal processors (digital signal processors), microcomputers, field programmable gate arrays (field programmable gate arrays, FPGAs), programmable logic units (programmable logic units, PLUs), microprocessors, or any other device capable of executing and responding to instructions (instructions). The processing device can execute an operating system (OS) and one or more application software executed in the operating system. In addition, the processing device responds to the execution of the software to access, store, operate, process and generate data. For ease of understanding, a method with only one processing device is described, but a person of ordinary skill in the art should understand that the processing device can include multiple processing elements (processing element) and / or multiple types of processing elements. For example, the processing device can include multiple processors or a processor and a controller. Furthermore, other processing configurations such as parallel processors can also be included.

[0190] Software can include a computer program, code, instructions, or a combination of more than one of these, that causes a processing device to operate in a desired manner, or that, alone or collectively, commands a processing device. Software and / or data can be permanently or temporarily embody any type of device, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave for interpretation by a processing device or for providing commands or data to a processing device. Software is distributed on computer systems connected by a network and can be stored or executed in a distributed manner. Software and data can be stored in one or more computer-readable storage media.

[0191] The method according to one embodiment is embodied in the form of program commands that can be executed by a variety of computer means and recorded in a computer-readable medium. The computer-readable medium can include program commands, data files, data structures, etc. in a single or combined form. The program instructions recorded in the medium can be instructions specially designed and constructed to implement the embodiment, or instructions that can be used based on known knowledge by ordinary technicians in the field of computer software. Computer-readable recording media can include magnetic media such as hard disks, floppy disks, and tapes; optical media similar to CD-ROMs, DVDs, etc.; magneto-optical media similar to floptical disks, and hardware devices specially constructed for storing and executing program commands similar to read-only memories (ROMs), random access memories (RAMs), flash memories, etc. Examples of program instructions include not only machine language codes generated by compilers, but also high-level language codes that can be executed by computers by using interpreters, etc.

[0192] To perform the operations of the embodiments, the hardware device may be configured in such a manner that the operations are implemented with one or more software modules, and vice versa.

[0193] Although the above embodiments are described based on limited embodiments and drawings, a person skilled in the art can make various modifications and changes based on the above description. For example, if the described techniques are performed in a different order than the described method, and / or if the components in the described system, architecture, device or circuit are combined or combined in different forms, or replaced or substituted by other components or equivalents, appropriate results can also be obtained.

[0194] Therefore, other implementations, other examples, and equivalents of the claims should be interpreted to be included within the scope of the appended claims.

Claims

1. A virtual smoke image output method, executed by an electronic device, characterized in that: include: receiving a connection request from the aerosol generating device in a state of being paired with one or more external controllers, the step of determining a type of controller of the aerosol generating device, When the determined controller type corresponds to a target external controller type among the one or more external controllers, the step of stopping pairing with the target external controller and connecting with the aerosol generating device is performed, the step of receiving sensed information about the state of the aerosol generating device, The step of generating a virtual smoke image based on the sensing information, and The step of outputting the virtual smoke image through the display of the electronic device.

2. The virtual smoke image output method according to claim 1, characterized in that: The step of outputting the virtual smoke image through the display of the electronic device includes: The step of superimposing the virtual smoke image on a reference image of an existing application being executed by the electronic device and outputting the result.

3. The virtual smoke image output method according to claim 1, characterized in that: The step of generating a virtual smoke image based on the sensing information includes: a step of determining a property of virtual smoke based on the sensing information, wherein the property of the virtual smoke includes at least one of size, shape and duration of the virtual smoke, and The step of generating the virtual smoke image based on the properties of the virtual smoke.

4. The virtual smoke image output method according to claim 1, characterized in that: The step of outputting the virtual smoke image through the display of the electronic device includes: the step of determining an exhalation moment at which a user of the aerosol generating device begins to exhale based on the sensed information, and The step of outputting the virtual smoke image at the exhalation moment.

5. The virtual smoke image output method according to claim 4, characterized in that: The step of determining an exhalation moment at which a user of the aerosol generating device starts to exhale based on the sensing information comprises: A step of determining the exhalation moment based on a first moment and a second moment sensed by the aerosol generating device, wherein the first moment is a moment when the distance between one side of the aerosol generating device and a part of the user's body reaches within a first threshold; and the second moment is a moment after the first moment when the distance between the one side of the aerosol generating device and the part of the user's body exceeds a second threshold.

6. The virtual smoke image output method according to claim 4, characterized in that: The step of determining an exhalation moment at which a user of the aerosol generating device starts to exhale based on the sensing information comprises: The step of determining the exhalation time based on a change in capacitance of the aerosol generating device.

7. The virtual smoke image output method according to claim 6, characterized in that: The step of determining the exhalation time based on the capacitance change of the aerosol generating device comprises: A step of determining the exhalation moment based on a first moment and a second moment sensed by the aerosol generating device, wherein the first moment is the moment when the value of the capacitance of the aerosol generating device reaches a first threshold; and the second moment is the moment when the value of the capacitance of the aerosol generating device exceeds a second threshold after the first moment.

8. The virtual smoke image output method according to claim 1, characterized in that: The connection request is received, via an adapter connected to the electronic device, from the aerosol generating device establishing a wireless communication channel with the adapter.

9. The virtual smoke image output method according to claim 1, characterized in that: The connection request is received from the aerosol generating device that directly establishes a wireless communication channel with the electronic device. 10 . A computer-readable recording medium storing a program for executing the virtual smoke image output method according to claim 1 .

11. An electronic device, characterized in that: include: monitor, A memory storing a program for outputting a virtual smoke image, and A processor, for executing the program; The processor performs the following steps: receiving a connection request from the aerosol generating device in a state of being paired with one or more external controllers, the step of determining a type of controller of the aerosol generating device, When the determined controller type corresponds to a target external controller type among one or more of the external controllers, the step of stopping pairing with the target external controller and connecting with the aerosol generating device is performed, the step of receiving sensed information about the state of the aerosol generating device, The step of generating a virtual smoke image based on the sensing information, and The step of outputting the virtual smoke image through the display of the electronic device.

Images