METHOD AND APPARATUS FOR OUTPUTTING VIRTURAL SMOKE IMAGE / VIDEO

Abstract

To output a virtual smoke image, the electronic device can output a target image content selected by a user via a display of the electronic device, receive detection information regarding the status of an aerosol generating device from an aerosol generating device connected to the electronic device, generate a virtual smoke image based on weather information and the detection information corresponding to the target image content, and output the virtual smoke image via the display of the electronic device.

Description

[Technical field]

[0001] The following embodiments relate to techniques for implementing virtual smoke in electronic devices. [Background technology]

[0002] In recent years, virtual reality, augmented reality, and mixed reality technologies that apply computer graphics technology have been developed. In this regard, virtual reality technology refers to a technology that uses a computer to create a virtual space that does not exist in the real world and makes the virtual space feel real, while augmented reality or mixed reality technology refers to a technology that adds and expresses information generated by a computer in the real world, that is, a technology that allows interaction with a user in real time by combining the real world and the virtual world.

[0003] Augmented reality and mixed reality technologies are being used in combination with technologies in various fields. In the field of electronic cigarettes, there is an increasing demand for devices that realize virtual reality or augmented reality services, and various research projects are being conducted in response to this demand. Summary of the Invention [Problem to be solved by the invention]

[0004] One embodiment may provide a method for outputting a virtual smoke image performed by an electronic device.

[0005] One embodiment may provide an electronic device for performing the method of outputting a virtual smoke image. [Means for solving the problem]

[0006] A virtual smoke image output method performed by an electronic device according to one embodiment may include an operation of outputting a target image content selected by a user through a display of the electronic device, an operation of receiving detection information regarding the status of an aerosol generating device from an aerosol generating device connected to the electronic device, an operation of generating a virtual smoke image based on weather information corresponding to the target image content and the detection information, and an operation of outputting the virtual smoke image through the display of the electronic device.

[0007] The target video content may be pre-created content that includes the weather information corresponding to a scene in the target video content.

[0008] The virtual smoke image may be output to be composited with the scene of the target image content.

[0009] The operation of generating a virtual smoke image based on weather information and the detection information corresponding to the target video content may include an operation of determining the weather information based on a scene in the target video content, and an operation of generating the virtual smoke image based on the weather information and the detection information.

[0010] The operation of generating a virtual smoke image based on weather information and the detection information corresponding to the target video content may include an operation of determining the weather information based on position information for the target video content, and an operation of generating the virtual smoke image based on the weather information and the detection information.

[0011] The operation of outputting the virtual smoke image through a display of the electronic device may include an operation of superimposing the smoke image on the target image content being output by the electronic device and outputting the smoke image.

[0012] The operation of generating a virtual smoke image based on the weather information and the detection information corresponding to the target video content may include an operation of determining attributes of virtual smoke based on the weather information and the detection information (the attributes of the virtual smoke include at least one of a size, a shape, and a duration of the virtual smoke), and an operation of generating the virtual smoke image based on the attributes of the virtual smoke.

[0013] The operation of outputting the virtual smoke image through the display of the electronic device may include an operation of determining an exhalation point at which a user of the aerosol generating device begins exhaling based on the detection information, and an operation of outputting the virtual smoke image at the exhalation point.

[0014] The operation of determining an exhalation time when a user of the aerosol generating device starts exhaling based on the detection information may include an operation of determining the exhalation time based on a first time when a distance between one side of the aerosol generating device and a part of the user's body reaches within a first threshold, sensed by the aerosol generating device, and a second time after the first time when the distance between the one side of the aerosol generating device and a part of the user's body exceeds a second threshold.

[0015] The operation of determining the exhalation time at which a user of the aerosol generating device begins to exhale based on the detection information may include an operation of determining the exhalation time based on a change in capacitance of the aerosol generating device.

[0016] The operation of determining the exhalation time based on a change in the capacitance of the aerosol generating device may include an operation of determining the exhalation time based on a first time at which a value of the capacitance of the aerosol generating device sensed by the aerosol generating device reaches within a first threshold and a second time after the first time at which the value of the capacitance of the aerosol generating device exceeds a second threshold.

[0017] The connection request may be received from the aerosol generating device that has established a wireless communication channel with a dongle connected to the electronic device via the dongle.

[0018] The electronic device and the aerosol generating device directly establish a wireless communication channel, and the detection information can be received from the aerosol generating device via the wireless communication channel.

[0019] An electronic device performing a method for outputting a virtual smoke image according to one embodiment includes a display, a memory having a program for outputting a virtual smoke image recorded therein, and a processor for executing the program, and the processor may perform operations of outputting a target image content selected by a user through the display, receiving detection information regarding the status of an aerosol generating device from an aerosol generating device connected to the electronic device, generating a virtual smoke image based on weather information corresponding to the target image content and the detection information, and outputting the virtual smoke image through the display of the electronic device. Effect of the Invention

[0020] A method may be provided for outputting a virtual smoke image via a display of an electronic device.

[0021] A method for outputting a virtual smoke image and video content reflecting weather information of the video content through a display of an electronic device may be provided. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a diagram for explaining a virtual smoke image output system according to an embodiment. [Diagram 2] 1 is an exemplary diagram of a display output screen of an electronic device in accordance with an embodiment. [Diagram 3]1A to 1C are diagrams illustrating a user's smoking behavior divided into sections and a method for predicting a user's exhalation time and outputting a virtual smoke image according to an embodiment of the present invention. [Figure 4] 1 is a flowchart of a virtual smoke image output method according to an embodiment. [Diagram 5] FIG. 1 illustrates an example of an aerosol generating device according to one embodiment. [Figure 6] 1 is a flowchart of a virtual smoke image output method according to an embodiment. [Figure 7] 1 is a flowchart of a virtual smoke image output method according to an embodiment. [Figure 8] 1 is a flowchart of a virtual smoke image output method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The specific structural or functional description of the embodiment is disclosed for the purpose of illustration only, and can be modified and implemented in various forms. Therefore, the actual implementation is not limited to the specific embodiment disclosed, and the scope of the present specification includes modifications, equivalents, or alternatives included in the technical ideas described in the embodiment.

[0024] Although terms such as first or second may be used to describe various components, these terms should be construed only for the purpose of distinguishing one component from another component, for example, a first component may be named a second component, and similarly, a second component may also be named a first component.

[0025] When a component is referred to as being "coupled" to another component, it may be directly coupled or connected to the other component, but it should be understood that there may be other components in between.

[0026] The singular term includes the plural term unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to specify the presence of a stated feature, number, step, operation, component, part, or combination thereof, and should not be understood to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted as meanings consistent with the meanings they have in the context of the relevant art, and should not be interpreted as ideal or overly formal meanings unless expressly defined in this specification.

[0028] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are used to designate the same components regardless of the drawing numerals, and duplicated description thereof will be omitted.

[0029] FIG. 1 is a diagram for explaining a virtual smoke image output system according to an embodiment.

[0030] 1, a system 100 for outputting a virtual smoke image can include an electronic device 110 for outputting a virtual smoke image, an aerosol generating device 120, and a dongle 130. A user 10 using the system 100 can wear the electronic device 110 and smoke via the aerosol generating device 120. "Smoking" can refer to heating, burning, and other things that cause the emission of smoke or aerosol from a smokable substance. The electronic device 110 according to one embodiment can realize virtual smoke through an image in conjunction with the smoking behavior of the user 10 via the aerosol generating device 120.

[0031] In one embodiment, the aerosol generating device 120 can be named an electronic cigarette device, an inhaler, or a smoking stick. The structure and specific operation method of the aerosol generating device 120 will be described in detail with reference to FIG. 5 below.

[0032] According to an embodiment, a user may use the aerosol generating device 120 to engage in a smoking act, which may generate smoke 142 when the user 10 breathes in. According to an embodiment, the aerosol generating device 120 may provide an aerosol to the user 10 by heating an aerosol product within the aerosol generating device 120 in various ways. For example, the aerosol product may be a stick that is inserted into the aerosol generating device 120. As another example, the aerosol product may be a replaceable cartridge that is inserted into the aerosol generating device 120.

[0033] According to an embodiment, visually observable smoke 142 may be generated by the breath of the user 10 who inhales the aerosol provided to the aerosol generating device 120, but is not limited thereto. For example, smoke 142 may not be generated by the breath of the user 10.

[0034] According to an example, the virtual smoke image may be output when the user smokes using the aerosol generating device 120 but smoke 142 is not generated. According to an example, the virtual smoke image may be output even when the user does not actually smoke using the aerosol generating device 120. For example, the virtual smoking program of the electronic device 110 may provide a specific mode that can generate and output a virtual smoke image based on detection information received from the aerosol generating device 120. As a result, even if smoke 142 is not visually observed in reality from the exhaled breath of the user 10 who smokes using the aerosol generating device 120, the user 10 can observe the virtual smoke image output by the electronic device 110 through virtual reality or augmented reality. However, the present disclosure is not limited thereto.

[0035] According to an embodiment, the electronic device 110 may provide a virtual reality service or an augmented reality service independently, or may provide a virtual reality service or an augmented reality service in conjunction with the aerosol generating device 120 or a third electronic device (e.g., a smartphone) wirelessly or wiredly connected to the electronic device 110. The structure and operation method of the electronic device 110 will be described in detail with reference to FIG. 2 below.

[0036] Virtual reality or augmented reality is considered to mean visual simulation that creates a virtual or augmented visual environment. As used herein, the term "virtual reality" includes simulated images that are generated for visualization by the smoker and may include a partially or fully simulated environment. The term "augmented reality" as used herein is also considered to include a combination of simulated images used to augment a real or "live" environment, where the live image is used in combination with a simulated image overlaid on the live image. As a result of virtual or augmented reality, a simulated virtual image can be provided or a live visual image can be augmented with a simulated augmented image to form an interactive environment that can be seen and felt with reality. In the following, for convenience of explanation, the space in which virtual reality or augmented reality is provided will be referred to as a virtual space.

[0037] According to an embodiment, the electronic device 110 is worn at a predetermined position (e.g., on the head) of a user and can provide (or output) an image to the user. For example, the electronic device 110 can be configured in the form of at least one of glasses, goggles, a helmet, or a hat, but is not limited thereto.

[0038] According to an embodiment, the aerosol generating device 120 and the electronic device 110 may be connected via a dongle 130. For example, the aerosol generating device 120 and the dongle 130 may be a pair. A user may connect the dongle 130 and the electronic device 110 by inserting the dongle 130 into an interface (e.g., a universal serial bus (USB) port) of the electronic device 110. The dongle 130 may receive power for operation from the electronic device 110. A wireless communication channel may be established between the dongle 130 and the aerosol generating device 120. For example, the wireless communication channel may be a wireless communication channel based on Bluetooth low energy (BLE), LoRa, EnOcean, radio frequency (RF) communication, Zigbee (registered trademark), or infrared communication.

[0039] According to an embodiment, the electronic device 110 can install a communication driver and a virtual smoking program for the dongle 130 using data stored in the dongle 130. The electronic device 110 and the dongle 130 can transmit and receive information via the communication driver. The electronic device 110 can execute the virtual smoking program to generate and output a virtual smoke image.

[0040] According to an embodiment, the dongle 130 may receive detection information related to the state of the aerosol generating device 120 from the aerosol generating device 120 via a wireless communication channel. For example, the detection information related to the state of the aerosol generating device 120 may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a part of the user's body), detection value information of the capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of the camera of the aerosol generating device 120, smoking patterns learned using the pressure sensor and button of the aerosol generating device 120, and camera data and pressure sensor data.

[0041] According to an embodiment, the data format of the detection information received by the dongle 130 from the aerosol generating device 120 is a wireless signal format, and the data format of the detection information may be converted from the wireless signal format to a USB signal format by the dongle 130. The electronic device 110 may receive the detection information converted into the USB signal format from the dongle 130. After the virtual smoking program is executed, a virtual smoke image may be generated based on the detection information received from the aerosol generating device 120. The virtual smoke image may be output via a display of the electronic device 110.

[0042] According to an embodiment, the aerosol generating device 120 and the electronic device 110 may be connected via a wireless communication channel. The wireless communication channel may be at least one of short-range communication and cellular communication. For example, the short-range communication may include at least one of Bluetooth (registered trademark), BLE, or wireless LAN (e.g., WiFi direct). For example, the cellular communication may include at least one of LTE (long term evolution), LTE-A (LTE Advance), 5G (or NR (new radio)), CDMA (code division multiple access), WCDMA (wideband CDMA) (registered trademark), or GSM (global system for mobile communication) (registered trademark). Alternatively, the aerosol generating device 120 and the electronic device 110 may be connected by wire based on a USB method. In the above embodiment, the virtual smoking program may be pre-stored in the electronic device 110.

[0043] The system 100 according to an embodiment may further include a playback device 140. For example, the playback device 140 is a device capable of providing sound to the user 10, and the method of providing the sound is not limited. The electronic device 110, the aerosol generating device 120, and the playback device 140 included in the system 100 work together to provide an improved smoking experience to the user 10 using the system 100. Although the electronic device 110 and the playback device 140 are illustrated as separate entities in FIG. 1, the electronic device 110 may include the playback device 140 according to an embodiment.

[0044] According to an embodiment, the electronic device 110 may include one or more auxiliary devices, such as an electroencephalogram (EEG) measurement module, a surface electromyogram (SEM) sensor, etc., to enhance the realism of the smoking behavior in the virtual space.

[0045] FIG. 2 is an exemplary diagram of a display output screen of an electronic device in accordance with one embodiment.

[0046] The description with reference to Fig. 1 may be equally applied to the description with reference to Fig. 2, and duplicated content may be omitted. Hereinafter, the structure and operation method of the electronic device 110 will be described in detail.

[0047] According to an embodiment, the electronic device 110 may include a communication unit, a processor, a memory, a display, and a USB port. The electronic device 110 may be a device for virtual reality, augmented reality, or a metaverse, but the embodiment is not limited thereto.

[0048] The communication unit is connected to the processor and the memory to transmit and receive data. The communication unit may be connected to another external device to transmit and receive data. Hereinafter, an expression of transmitting and receiving "A" may indicate transmitting and receiving "information or data indicating A".

[0049] The communication unit may be realized as a circuitry within the electronic device 110. For example, the communication unit may include an internal bus and an external bus. As another example, the communication unit may be an element that connects the electronic device 110 to an external device (e.g., the dongle 130). The communication unit may be an interface. The communication unit may receive data from the external device (e.g., the dongle 130) and transmit the data to the processor and the memory.

[0050] The processor processes data received by the communication unit and data stored in the memory. A "processor" may be a data processing device implemented in hardware having circuits with a physical structure for performing desired operations. For example, the desired operations may include code or instructions included in a program. For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an application-specific integrated circuit (ASIC), or a field programmable gate array (FPGA).

[0051] The processor executes computer readable code (e.g., software) stored in the memory and instructions triggered by the processor, for example, the processor may execute instructions that cause a virtual smoke image to be output via a display.

[0052] The memory stores data received by the communication unit and data processed by the processor. For example, the memory may store a program, application, or software (e.g., a communication driver and a virtual smoking program). The stored program may be a set of syntax that is coded to output a virtual smoke image and is executable by the processor.

[0053] According to one embodiment, the memory may include one or more of volatile memory, non-volatile memory, and Random Access Memory (RAM), flash memory, a hard disk drive, and an optical disk drive.

[0054] The memory stores a set of instructions (e.g., software) that causes the electronic device 110 to operate. The set of instructions that cause the electronic device 110 to operate are executed by the processor.

[0055] According to one embodiment, the electronic device 110 and the aerosol generating device 120 may be connected via a dongle 130. For example, the dongle 130 may be connected to a USB port of the electronic device 110. A communication driver and a virtual smoking program stored in the dongle 130 may be installed in the electronic device 110 via the dongle 130.

[0056] According to one embodiment, when the dongle 130 is coupled (or inserted or connected) to a USB port of the electronic device 110, the communication driver and the virtual smoking program may be automatically installed in the electronic device 110. For example, the communication driver and the virtual smoking program may be installed or stored in the memory.

[0057] According to one embodiment, after the dongle 130 is connected to the USB port of the electronic device 110, if a separate input is received from the user, a communication driver and a virtual smoking program may be installed.

[0058] According to an embodiment, the processor can execute a virtual smoking program. For example, the virtual smoking program can be automatically executed after the virtual smoking program is installed. As another example, the virtual smoking program can be executed when a smoking start signal is received from a user or a smoking start signal is received from the aerosol generating device 120. The processor can generate a virtual smoke image based on detection information regarding the state of the aerosol generating device 120 received from the aerosol generating device 120 via the dongle 130.

[0059] The electronic device 110 may provide a virtual reality, augmented reality, or metaverse image in which tobacco smoke is generated to a user through a display. In one embodiment, the display may be controlled to be transparent or opaque depending on the intended use. When the electronic device 110 provides a virtual reality or metaverse image, the display may be controlled to be opaque, and when the electronic device 110 provides an augmented reality image, the display may be controlled to be transparent.

[0060] For example, the display may be made of, but is not limited to, a plastic material such as polycarbonate, or a glass material, and may be coated with at least one of a light reflective and anti-glare coating, an anti-fog coating, and an ultraviolet blocking coating.

[0061] According to an example, the system 100 may further include an article. The electronic device 110 may receive content information related to the article from the article. The electronic device 110 may change an image displayed on the display based on the content information related to the article.

[0062] In one embodiment, an image showing tobacco smoke may be displayed on the display, and the electronic device 110 may change the image relating to at least one of the color of the tobacco smoke, the amount of tobacco smoke, and the pattern of the tobacco smoke displayed on the display based on the content information relating to the item.

[0063] The electronic device 110 may further include a camera (not shown) that can scan the actual surroundings when the user uses the electronic device 110 to assist the user in using the electronic device 110.

[0064] The electronic device 110 may be installed with applications to provide a variety of experiences to the user. For example, a smoking application may be installed to provide a virtual space smoking system to provide a user with a smoking experience close to reality. However, the present disclosure is not limited thereto.

[0065] The electronic device 110 may further include a sensor for detecting or predicting a user's smoking behavior in the system 100. For example, the sensor may include a surface electromyogram sensor or an electroencephalogram measuring module attached to the user's skin to detect the user's smoking behavior. The electronic device 110 may include a communication device for communicating with the article. However, the present disclosure is not limited thereto.

[0066] 2, a target video content 200 selected by a user is output via a display of an electronic device 110. For example, the target video content may be smoking-specific video content that is pre-created to provide a user with virtual smoking. As another example, the target video content may not be smoking-specific video content, but may be any pre-created video. As yet another example, the target video content may be a real-time video.

[0067] In an embodiment, a virtual smoke image may be generated based on meteorological information corresponding to the target image content 200 and detection information regarding the state of the aerosol generating device 120. For example, the meteorological information corresponding to the target image content may include one or more of wind speed, wind direction, precipitation, temperature, humidity, fine dust, ultraviolet rays, sunset time, and day / night. For example, the meteorological information corresponding to the target image content 200 of FIG. 2 may include wind direction (right side), precipitation (e.g., 100 mm), and humidity (e.g., 90%), and the detection information regarding the state of the aerosol generating device 120 may include smoking pattern and pressure sensor data. Thus, the virtual smoke image may be generated based on the meteorological information and detection information such that the smoke in the image is scattered to the right side and has a thick concentration and a dark color.

[0068] A method for providing a virtual smoke image to a user will be described in detail with reference to the following FIGS.

[0069] FIG. 3 is a diagram illustrating a user's smoking behavior divided into sections and a method for predicting a user's exhalation time and outputting a virtual smoke image according to an embodiment.

[0070] The description with reference to FIGS. 1 and 2 may be equally applied to the description with reference to FIG. 3, and duplicate content may be omitted.

[0071] According to an embodiment, a user performs one smoking session from time t1 to time t4.

[0072] More specifically, referring to FIG. 3, a user according to an embodiment may inhale smokable material (e.g., aerosol) through the aerosol generating device 120 in a section D1 between time t1 and time t2. The aerosol generating device 120 may generate aerosol in various ways. Hereinafter, the user's action in section D1 may be referred to as a "smoke inhalation" action, and the user may inhale smokable material by biting the aerosol generating device 120 with his / her mouth and collecting it in his / her mouth through the smoke inhalation action. In section D1, the user may be in contact with one side of the aerosol generating device 120 (e.g., the inhalation port or mouthpiece).

[0073] In one embodiment, the user may inhale the smoke collected in the mouth in section D2 between time t2 and time t3 by inhaling. Hereinafter, the user's action in section D2 may be referred to as an "inhale" action (or an "inhale and lower hand" action), and the user inhales while separating one side of the aerosol generating device 120 from the lips and lowering the hand. In section D2, the user and the aerosol generating device 120 may be in a separated state.

[0074] In one embodiment, the user can exhale the inhaled smoke during the D3 period between time t3 and time t4. Hereinafter, the user's action during the D3 period may be referred to as an "exhale" action.

[0075] The length (inhalation time) of the D2 section according to an embodiment can be estimated based on the length (smoke inhalation time) of the D1 section. As an example, the length (inhalation time) of the D2 section can be proportional to the length (smoke inhalation time) of the D1 section. For example, when the smoke inhalation time is the first, the inhalation time can also be the first, and when the smoke inhalation time is the third, the inhalation time can also be the third. Alternatively, the inhalation time corresponding to the smoke inhalation time can be predicted using an artificial neural network. For example, the smoke inhalation time can be input to an artificial neural network trained based on paired learning data of the smoke inhalation time and the inhalation time to predict the inhalation time corresponding thereto. However, the method of estimating the inhalation time based on the smoke inhalation time is not limited to the above example.

[0076] Being able to predict the length of D2 (inhalation time) means being able to predict t3 (exhalation time). In other words, the length of D2 can be predicted by measuring the length of D1 (smoke inhalation time), and t3 (exhalation time) can be predicted by adding the predicted length of D2 to the measured length of D1.

[0077] According to an embodiment, the length of the D3 section (exhalation time) may be determined based on at least one of the length of the D1 section (smoke inhalation time) and the length of the D2 section (inhalation time). For example, the length of the D3 section may be determined to be proportional to the length of the D1 section or the length of the D2 section. The t4 time point may be predicted based on the predicted t3 (exhalation time point) and the length of the D3 section.

[0078] According to an embodiment, a user can set information regarding the length of section D3 in advance in electronic device 110. For example, a ratio to the length of section D1 or the length of section D2 can be set by the user. A user who wishes to maintain a relatively long output time of virtual smoke can set the ratio to be large.

[0079] In the virtual smoking system according to an embodiment, a user can experience virtual reality or augmented reality of smoking by linking smoking in the real world through the aerosol generating device 120 with the electronic device 110. More specifically, the electronic device 110 according to an embodiment can realize virtual smoke in the virtual space at the time when the user exhales real smoke through the aerosol generating device 120, thereby realizing virtual smoke that does not feel foreign.

[0080] A user of the aerosol generating device 120 according to an embodiment repeats the actions D1 to D3 at least once to perform a smoking act, and the electronic device 110 or the aerosol generating device 120 can detect or predict the user's inhalation, inhalation, and exhalation of tobacco smoke to realize virtual smoke. Hereinafter, "realizing virtual smoke" may include determining at least one of the output time of the virtual smoke, i.e., the exhalation time, and the attributes of the virtual smoke. For example, the attributes of the virtual smoke may include one or more of the size, shape, duration, volume, density, and color of the virtual smoke. This is not limited in the present specification.

[0081] Referring to the flowchart of FIG. 3, when a user starts smoking according to an embodiment 300, the user can inhale smokable material through the aerosol generating device 120 and collect it in the mouth. At this time, the time (e.g., D1) during which the user inhales tobacco smoke is measured 310. After that, when the user finishes inhaling smoke, the user performs an inhalation and hand-down action. At this time, the time (e.g., D2) during which the user inhales and hands-down can be measured 320. The electronic device 110 or the aerosol generating device 120 can predict the time during which the user inhales and hands-down and the time of exhalation (e.g., t3) at which the user starts exhaling based on the smoke inhalation time 310. When the user starts exhaling, the electronic device 110 can realize 340 virtual smoke during the time (e.g., D3) during which the user exhales. When a predetermined time has elapsed since the user started exhaling, the electronic device 110 can end 350 the virtual smoke output.

[0082] FIG. 4 is a flowchart of a virtual smoke image output method according to an embodiment.

[0083] The following operations 410-440 are performed by the electronic device 110 described above with reference to FIG.

[0084] At operation 410, the electronic device 110 may output the targeted video content selected by the user via a display of the electronic device 110.

[0085] In one embodiment, the target video content can be selected and output by the user from within an existing application on the electronic device 110 .

[0086] In one embodiment, the target video content may be pre-created content including weather information corresponding to a scene in the target video content, i.e., smoking-only video content. For example, the weather information may be stored in advance in a database for a number of items such as wind speed, wind direction, precipitation, temperature, and humidity. For example, the smoking-only video content may be provided by a virtual smoking program.

[0087] In one embodiment, the target video content may be any video content other than smoking-specific video content, for example, video content posted on websites such as YouTube and Naver.

[0088] In one embodiment, the target video content may be real-time video content rather than smoking-specific video content. For example, the real-time video may be provided through an open API (Open Application Programming Interface) or public API service, such as CCTV video from Naver Maps, Google Maps, or OpenStreetMap.

[0089] In one embodiment, the electronic device 110 may receive a connection request from the aerosol generating device 120. When the electronic device 110 receives the connection request from the aerosol generating device 120, the target video content may be output via the display 200 of the electronic device 110.

[0090] In one embodiment, the electronic device 110 may receive a connection request from the aerosol generation device 120 via the dongle 130. Specifically, the connection request may be received from the aerosol generation device 120 that has established a wireless communication channel with the dongle 130 via the dongle 130 connected to the electronic device 110. For example, the wireless communication channel may be a wireless communication channel based on BLE, LoRa, EnOcean, RF communication, Zigbee (registered trademark), or infrared communication.

[0091] In one embodiment, the electronic device 110 may receive a connection request from the aerosol generation device 120 via a wireless communication channel. For example, the connection request may be received from the aerosol generation device 120 that has directly established a wireless communication channel with the electronic device 110. The wireless communication channel may be at least one of short-range communication and cellular communication. As another example, the aerosol generation device 120 and the electronic device 110 may be connected by wire based on a USB method. In the above embodiment, the virtual smoking program may be pre-stored in the electronic device 110.

[0092] In one embodiment, the electronic device 110 may receive a connection request between the aerosol generating device 120 and the electronic device 110 directly from a user via a user interface (UI) of the electronic device 110.

[0093] According to one embodiment, if the target video content is not smoking-specific video content and is output by an existing application, the following operations 420-440 are performed while the target video content is being output by the existing application in the electronic device 110. That is, the existing application and the virtual smoking program can be executed in parallel at the same time.

[0094] In operation 420, the electronic device 110 may receive detection information related to the state of the aerosol generating device 120 from the aerosol generating device 120 coupled to the electronic device 110. For example, the detection information may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a body part of a user), detection value information of a capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of a camera of the aerosol generating device 120, smoking patterns learned using a pressure sensor and a button of the aerosol generating device 120, camera data, and pressure sensor data. The detection information of the aerosol generating device 120 will be described in further detail with reference to FIG. 5 below.

[0095] At operation 430, the electronic device 110 may generate a virtual smoke image based on the weather information corresponding to the target image content and the detected information regarding the status of the aerosol generating device 120.

[0096] In one embodiment, operation 430 is performed by executing a virtual smoking program on the electronic device 110 .

[0097] In one embodiment, the electronic device 110 may determine attributes of the virtual smoke based on weather information and detection information. For example, the electronic device 110 may determine attributes of the virtual smoke based on weather information, which may include one or more of wind speed, wind direction, precipitation, temperature, humidity, and day / night, and detection information for virtual smoking described in operation 420. For example, the attributes of the virtual smoke may include one or more of the size, shape, duration, volume, density, and color of the virtual smoke. This is not limited in this specification. The electronic device 110 may generate a virtual smoke image based on the attributes of the virtual smoke.

[0098] At operation 440 , the virtual smoke image may be output via a display of the electronic device 110 .

[0099] In one embodiment, the electronic device 110 can determine the exhalation time based on a first time at which the distance between one side of the aerosol generating device 120 and a part of the user's body reaches within a first threshold, as sensed by the aerosol generating device 120, and a second time after the first time at which the distance between one side of the aerosol generating device 120 and a part of the user's body exceeds a second threshold.

[0100] In one embodiment, the electronic device 110 can determine the exhalation time based on a first time at which the capacitance value of the aerosol generating device 120 sensed by the aerosol generating device 120 reaches within a first threshold and a second time after the first time at which the capacitance value of the aerosol generating device 120 exceeds a second threshold.

[0101] In one embodiment, when the target video content is a pre-created content including weather information corresponding to a scene in the target video content, i.e., a video content for smoking, the virtual smoke video may be output so as to be composited with the scene in the target video content. For example, the video may be output so that the virtual smoke appears in the scene in the target video content.

[0102] In one embodiment, when the target video content is not smoking-specific video content but is any video content or real-time video content, the virtual smoke video may be output by being superimposed on the target video content being output by the electronic device 110. For example, the electronic device 110 may output an image by superimposing the image using a layer for outputting the target video content and a virtual smoke video layer. The user may observe the superimposed image.

[0103] Even if a user cannot visually observe smoke in reality, the user can observe smoke through an image in which virtual smoke appears, and can satisfy the sensation of smoking through the observed smoke.

[0104] FIG. 5 is a diagram illustrating an aerosol generating device according to an embodiment.

[0105] 5, the aerosol generating device 500 (e.g., the aerosol generating device 120 of FIG. 1) can have a cigarette shape. For example, the length and diameter of the aerosol generating device 500 can be substantially the same as the length and diameter of a traditional cigarette-type cigarette, but are not limited to the described embodiment. The aerosol generating device 500 can be divided into a first end 510, a middle portion 520, and a second end 530.

[0106] The first end 510 can be a portion for replicating the lighting portion of a cigarette. The first end 510 can include an LED. For example, the first end 510 can include a red LED.

[0107] The second end 530 may include nicotine. In one embodiment, the surface of the second end 530 may be coated with nicotine, or the second end 530 may include an aerosol product inside. For example, when the aerosol product is heated, an aerosol containing nicotine may be generated. When a user bites the second end 530 of the aerosol generating device 500 and inhales the aerosol, nicotine may be provided to the user. Meanwhile, the second end 530 may further include a flavoring portion containing a flavoring in addition to the aerosol product.

[0108] In one embodiment, the intermediate portion 520 serves to connect the first end 510 and the second end 530. The length of the intermediate portion 520 determines the overall length of the aerosol generating device 500, and the length of the intermediate portion 520 can be set during manufacturing of the aerosol generating device 500 so that the length of the aerosol generating device 500 corresponds to the length of a typical cigarette.

[0109] In another embodiment, the aerosol generating device 500 may include other electronic devices or electronic circuits inside (e.g., the middle part 520) as an electronic device. For example, the aerosol generating device 500 may include a sensor for acquiring detection information regarding the state of the aerosol generating device 500 or detecting a user's smoking behavior. The sensor may include, for example, a sensor for detecting a change in distance, a sensor for detecting a change in orientation, a sensor for detecting a change in capacitance, and a pressure sensor. The aerosol generating device 500 may include other electronic devices outside the housing as an electronic device. For example, the aerosol generating device 500 may include a camera for observing a user's smoking behavior. The aerosol generating device 500 may have buttons inside or outside for a power adjustment function, a smoking pattern learning function, a smoking detection function, or a control function of the electronic device 110. The aerosol generating device 500 may include a communication device for communicating with an XR device (e.g., the electronic device 110 in FIG. 1) as an electronic device. However, the present disclosure is not limited to an aerosol generating device including a sensor, a camera, or a button. Hereinafter, a method for determining an expiration time point using different detection information will be described, and the same or similar content may be omitted.

[0110] <Distance information from distance sensor> The distance detection sensor according to an embodiment may be located inside or outside the second end 530. The distance detection sensor according to an embodiment may be an infrared type, a lidar type, an ultrasonic type, or the like. However, the distance detection sensor of the present disclosure is not limited to the above types. Hereinafter, a method for determining an exhalation time when a user starts exhaling using the distance detection sensor according to an embodiment will be described in detail.

[0111] When a user according to an embodiment starts smoking, the distance detection sensor may detect the distance between an object (e.g., the user's lips) and the aerosol generating device 500. The aerosol generating device 500 may transmit detection information of the distance detection sensor to the electronic device 110 via a dongle (e.g., the dongle 130 in FIG. 1 ). The electronic device 110 may determine the time of exhalation based on the detection information of the distance detection sensor received from the aerosol generating device 500.

[0112] In one embodiment, the act of starting to smoke includes an act of the user bringing the aerosol generating device 500 to their lips and inhaling. The user can turn on the aerosol generating device 500 to start heating the aerosol product in the aerosol generating device 500, and the turned-on aerosol generating device 500 can preheat a heater that heats the aerosol product.

[0113] According to an embodiment, the operation of the distance detection sensor detecting the distance between the object (e.g., a body part of a user) and the aerosol generating device 500 may include an operation of detecting a first time when the object contacts the second end 530 of the aerosol generating device 500 and an operation of detecting a second time when the object moves away from the second end 530 of the article. For example, the operation of detecting the first time may include the distance detection sensor detecting that the second end 530 of the aerosol generating device 500 and the object approach each other to a distance within a first threshold. For example, the operation of detecting the second time may include the distance detection sensor detecting that the object moves away from the second end 530 of the aerosol generating device 500 to a distance exceeding a second threshold after the object contacts the second end 530 of the aerosol generating device 500.

[0114] According to an embodiment, the operation of determining an exhalation time when a user starts exhaling includes an operation of determining an exhalation time based on an interval between a first time and a second time. The first time may be a time when the user starts smoke inhalation. The interval between the first time and the second time is a time when the user takes smoke inhalation, and a first breathing time of the user may be predicted based on the smoke inhalation time. The second time may be a time when the user starts inhalation. A second breathing time may be predicted based on the interval between the first time and the second time and the first breathing time. The second breathing time may be an exhalation time of the user. For example, the second breathing time may be determined as an exhalation time.

[0115] According to an embodiment, since the smoke inhalation time of the user is the interval between the first time and the second time, the first breathing time, which is the time when the user inhales, can be estimated similarly to the smoke inhalation time. Thereafter, the electronic device 110 can determine the time of the second breathing, which is the time when the user starts exhaling, as the time when the first breathing time has elapsed from the second time. For example, if the time from the first time when the object arrives at the second end 530 to the second time when the object is removed from the second end 530 is 1 second, the first breathing time can be estimated to be 1 second, and the time when the user inhales can also be predicted to be 1 second similarly to the first breathing time. As a result, the time of the second breathing, which is the time when the first breathing time has elapsed from the second time, can be predicted to be the time when 1 second has elapsed since the object was separated from the aerosol generating device 500. As a result, the electronic device 110 can determine the time when 1 second has elapsed since the object was removed as the time of exhalation.

[0116] According to an embodiment, the operation of determining the exhalation time point based on the distance detection information includes an operation of estimating the inhalation amount and the respiration amount of the user based on the interval between the first time point and the second time point. The inhalation amount of the user may include the inhalation amount of the aerosol inhaled by the user through the aerosol generating device 500 in a state where the aerosol generating device 500 is in contact with an object. The respiration amount of the user may include the inspiration amount inhaled by the user in a state where the aerosol generating device 500 is not in contact with an object. The exhalation time point may be determined based on the estimated respiration amount and the inhalation amount.

[0117] The operation of determining the time of exhalation based on the detection information according to an embodiment includes an operation of determining whether the aerosol generating device 500 satisfies a predetermined operating condition, and an operation of determining the time of exhalation based on the determination that the aerosol generating device 500 satisfies the operating condition. The predetermined operating condition includes at least one of whether the user turns on / off the power of the aerosol generating device 500, and whether a puff sensor and a pressure sensor included in the aerosol generating device operate. Although the detection value of the distance detection sensor may change depending on other objects (e.g., a finger, etc.) instead of the predetermined object, the distance detection sensor can be activated only when the user turns on / off the power, and the distance detection sensor can be activated by detecting the inhalation of the user through a puff sensor or a pressure sensor, etc. However, the inhalation sensor of the present disclosure is not limited to a puff sensor or a pressure sensor.

[0118] The operation of transmitting the detection information to the electronic device 110 via the dongle 130 according to an embodiment may include an operation of transmitting information on the type of virtual smoke and an operation of transmitting information on a delay time (delay) occurring in communication between the aerosol generating device 500 and the electronic device 110. The type of virtual smoke may be changed according to a user's selection, or may be changed by an aerosol product or other flavoring unit built into the second end 530. The detection information may also include information on the user's smoking amount, breathing amount, etc., and the present disclosure is not limited thereto. If the exhalation time does not match the user's actual smoking behavior or an accurate predicted time cannot be grasped, additional operations may be performed to perform a separate "exhalation prediction detection" processing process.

[0119] <Detection value information of capacitance sensor> The capacitance sensor according to an embodiment may be located inside or outside the second end 530. The capacitance sensor according to an embodiment may be a sensor based on a capacitor. However, the capacitance sensor of the present disclosure is not limited to the above method. Hereinafter, a method for determining an exhalation time when a user starts exhaling using the capacitance sensor according to an embodiment will be described in detail.

[0120] When a user according to an embodiment starts smoking, the capacitance sensor may detect the capacitance of the capacitance sensor, which changes when an object (e.g., the user's lips) comes into contact with the aerosol generating device 500. For example, the capacitance sensor may detect a change in capacitance. The aerosol generating device 500 may transmit the detection information of the capacitance sensor to the electronic device 110 via the dongle 130. The electronic device 110 may determine the time of exhalation based on the detection information of the capacitance sensor received from the aerosol generating device 500.

[0121] In one embodiment, the act of starting smoking includes an act of the user bringing the aerosol generating device 500 into contact with the lips and inhaling. The user can turn on the aerosol generating device 500 to start heating the aerosol product in the aerosol generating device 500, and the turned-on aerosol generating device 500 can preheat a heater that heats the aerosol product.

[0122] According to an embodiment, the operation of detecting a change in capacitance when the object comes into contact with the aerosol generating device 500 by the capacitance sensor may include an operation of detecting a first time when the object comes into contact with the second end 530 of the aerosol generating device 500 and an operation of detecting a second time when the object moves away (or is removed) from the second end 530 of the article. For example, the operation of detecting the first time may include the capacitance sensor detecting that the capacitance detected when the object comes into contact with the second end 630 of the aerosol generating device 500 reaches within a first threshold value. For example, the operation of detecting the second time may include the capacitance sensor detecting that the capacitance detected when the object comes into contact with the second end 530 of the aerosol generating device 500 and then is removed exceeds a second threshold value.

[0123] The operation of the electronic device 110 according to an embodiment to determine the exhalation time based on the capacitance detection information includes an operation of determining the exhalation time based on an interval between a first time and a second time. The first time may be a time when the user starts inhaling smoke. The interval between the first time and the second time is a time when the user inhales smoke, and a first breathing time of the user may be predicted based on the smoke inhalation time. The second time may be a time when the user starts inhaling. A second breathing time may be predicted based on the interval between the first time and the second time and the first breathing time. The second breathing time may be a time when the user starts exhaling. For example, the second breathing time may be determined as the exhalation time.

[0124] According to an embodiment, since the smoke inhalation time of the user is the interval between the first time and the second time, the first breathing time, which is the time when the user inhales, can be estimated similarly to the smoke inhalation time. Thereafter, the electronic device 110 can determine the time of the second breathing, which is the time when the user starts exhaling, as the time when the first breathing time has elapsed from the second time. For example, if the time from the first time when the object arrives at the second end 530 to the second time when the object is removed from the second end 530 is 1 second, the first breathing time can be estimated as 1 second, and the time when the user inhales can also be predicted as 1 second similarly to the first breathing time. As a result, the time of the second breathing, which is the time when the first breathing time has elapsed from the second time, can be predicted as the time when 1 second has elapsed since the object was separated from the aerosol generating device 500. As a result, the electronic device 110 can determine the time when 1 second has elapsed since the object was removed as the time of exhalation.

[0125] According to an embodiment, the operation of determining the exhalation time point based on the capacitance detection information includes an operation of estimating the inhalation amount and the breathing amount of the user based on the interval between the first time point and the second time point. The inhalation amount of the user may include the inhalation amount of the aerosol inhaled by the user through the aerosol generating device 500 in a state where the aerosol generating device 500 is in contact with an object. The breathing amount of the user may include the inhalation amount of the user inhaled in a state where the aerosol generating device 500 is not in contact with an object. The exhalation time point may be determined based on the estimated breathing amount and the inhalation amount.

[0126] The operation of the electronic device 110 according to an embodiment to determine the time of exhalation based on the capacitance detection information includes an operation of determining whether the aerosol generating device 500 satisfies a predetermined operating condition, and an operation of determining the time of exhalation based on the determination that the aerosol generating device 500 satisfies the operating condition. The predetermined operating condition includes at least one of whether the user turns on / off the power of the aerosol generating device 500, and whether the puff sensor and the pressure sensor included in the aerosol generating device operate. Although the detection value of the capacitance sensor may change depending on other objects (e.g., a finger, etc.) instead of the predetermined object, the capacitance sensor can be activated only when the user turns on / off the power, and the capacitance sensor can be activated by detecting the inhalation of the user via the puff sensor or the pressure sensor, etc. However, the inhalation sensor of the present disclosure is not limited to the puff sensor or the pressure sensor.

[0127] The operation of transmitting the detection information of the capacitance sensor to the electronic device 110 via the dongle 130 according to an embodiment includes an operation of transmitting information about the type of virtual smoke and an operation of transmitting information about a delay time occurring in communication between the aerosol generating device 500 and the electronic device 110. The type of virtual smoke may be changed according to a user's selection, or may be changed by an aerosol product or other flavoring unit built into the second end 530. The detection information may also include information about the user's smoking amount, breathing amount, etc., and the present disclosure is not limited thereto.

[0128] <Orientation change information> The orientation change sensor according to an embodiment may be located inside or outside the second end 530. The orientation change sensor according to an embodiment may be a sensor such as a gyro sensor, an acceleration sensor, or a 3dof / 6dof sensor. However, the orientation change sensor of the present disclosure is not limited to the above method. Hereinafter, a method for determining an exhalation time when a user starts exhaling using the orientation change sensor according to an embodiment will be described in detail.

[0129] When a user in one embodiment starts smoking, the orientation change sensor can detect an orientation change between an object (e.g., the user's lips) and the aerosol generating device 500. The aerosol generating device 500 can transmit the detection information of the orientation change sensor to the electronic device 110 via the dongle 130. The electronic device 110 can determine the exhalation time based on the detection information of the orientation change sensor.

[0130] According to an embodiment, the operation of the orientation change sensor detecting the orientation change between the object and the aerosol generation device 500 may include an operation of detecting a first time when the object contacts the second end 530 of the aerosol generation device 500 and an operation of detecting a second time when the object moves away from the second end 530 of the article. For example, the operation of detecting the first time may include the orientation change sensor detecting that the second end 530 of the aerosol generation device 500 and the object approach each other in an orientation within a first threshold, and detecting a change in acceleration while the user brings the aerosol generation device 500 with the object. For example, the operation of detecting the second time may include detecting that the object moves away from the object in an orientation exceeding the second threshold after the object contacts the second end 530 of the aerosol generation device 500, and detecting a change in acceleration while the user removes the aerosol generation device 500 from the object.

[0131] According to an embodiment, the orientation change can be measured by comparing the x, y, and z-axis orientations of the aerosol generating device 500 when the user lowers his / her hand with the x, y, and z-axis orientations of the aerosol generating device 500 when the user brings the aerosol generating device 500 into contact with an object. When the user moves his / her hand to move the aerosol generating device 500, the measured acceleration can show a pattern of increasing, decreasing, and stopping. The orientation change can be predicted by measuring the change section of the acceleration, and the user's smoking time can be estimated based on the predicted orientation change.

[0132] According to an embodiment, the electronic device 110 determines the time of expiration based on the detection information of the orientation change, and determines the time of expiration based on the interval between the first time and the second time. Hereinafter, the process of determining the time of expiration is the same as that using the capacitance sensor, and therefore, a description thereof will be omitted.

[0133] <Camera monitoring information> The camera according to an embodiment may be located on the surface of the housing of the aerosol generating device 500. Hereinafter, a method for determining the exhalation time when a user starts exhaling using the camera according to an embodiment will be described.

[0134] When a user in one embodiment starts smoking, the camera can monitor the movement of an object (e.g., the user's lips). The aerosol generating device 500 can transmit the camera monitoring information (i.e., detection information) to the electronic device 110 via the dongle 130. The electronic device 110 can determine the exhalation time based on the camera monitoring information.

[0135] In one embodiment, the operation of the camera monitoring the distance between the object and the aerosol generating device 500 may include an operation of detecting a first time when the motion of the object changes due to contact of the object with the aerosol generating device 500 at the second end 530 of the aerosol generating device 500, and an operation of detecting a second time when the motion of the object changes due to separation of the object from the aerosol generating device 500 from the second end 530 of the article. For example, the operation of detecting the first time may include the camera detecting a first motion of the object bringing the second end 530 of the aerosol generating device 500 to the lips and touching it. For example, the operation of detecting the second time may include the camera detecting a second motion of the object being separated from the second end 530 of the aerosol generating device 500.

[0136] According to an embodiment, the electronic device 110 determines the time of exhalation based on the monitoring information of the camera, and determines the time of exhalation based on the interval between the first time and the second time. Hereinafter, the process of determining the time of exhalation is the same as that using the capacitance sensor, and therefore, a description thereof will be omitted.

[0137] <Smoking patterns learned using pressure sensors and buttons> A pressure sensor and a button (not shown) according to an embodiment may be included in the aerosol generating device 500. The pressure sensor may be located inside or outside the second end 530. The pressure sensor may measure the pressure generated when the user brings the second end to the lips or bites it to smoke, and may detect whether the user is inhaling. The button may have a function of turning on / off the power of the aerosol generating device 500, and may include an input function for storing the user's smoking pattern. The button may include a function of determining whether the user's input is an operation of the power function or an operation for storing the smoking pattern. For example, the button may have a function of turning on / off the power when pressed for a long time, a function of storing the smoking pattern when pressed for a short time, or a function of transmitting detection information such as changing the type of virtual smoke. However, the button of the present disclosure is not limited to the above functions.

[0138] According to an embodiment, when a user starts smoking, the aerosol generating device 500 can learn the smoking pattern using the pressure sensor and the button. The electronic device 110 can determine the exhalation time based on the learned smoking pattern and determine the volume and concentration of the virtual smoke. The aerosol generating device 500 can transmit the smoking pattern (i.e., the detection information) to the electronic device 110 via the dongle 130.

[0139] An operation of learning a smoking pattern using a pressure sensor and a button in one embodiment may include an operation of detecting a first time when an object contacts the second end 530 of the aerosol generating device 500 and the pressure detection value of the pressure sensor is measured to exceed a first threshold, an operation of detecting a second time after the first time when the object is separated from the second end 530 and the pressure detection value of the pressure sensor is measured to be within the second threshold, and an operation of detecting a third time after the second time when the button of the aerosol generating device 500 is operated.

[0140] In the method of learning a smoking pattern according to an embodiment, when a user starts puffing to smoke, the time when the pressure changes by the pressure sensor is measured and stored. Then, the time when the detection value of the pressure sensor is restored at the time when the user stops puffing is stored. The time duration between the start and end of puffing is the smoke inhalation time of the user, which is similar to the user's inhalation time. Then, when the user exhales the smoke he or she has inhaled, the user presses a button of the aerosol generating device 500 to input that exhalation is to begin. When the user's exhalation ends, the user stops pressing the button. The time difference between the start and end of exhalation is similar to the user's exhalation time. The aerosol generating device 500 can generate an algorithm for the user's smoking pattern based on the above-mentioned smoking pattern of the user performed at least once, and can store the smoking pattern learned based on the algorithm.

[0141] In one embodiment, storing a smoking pattern includes storing the smoking pattern based on intervals between a first time and a second time and between the second time and a third time, and collecting data regarding at least one or more intervals between the first time and the second time and between the second time and the third time.

[0142] The operation of determining the time of exhalation based on the learned smoking pattern by the electronic device 110 according to an embodiment includes an operation of determining the time of exhalation based on the interval between the first time and the second time. Hereinafter, the process of determining the time of exhalation is the same as that using the capacitance sensor, and therefore a description thereof will be omitted.

[0143] In one embodiment, determining the volume and density of virtual smoke based on the learned smoking pattern may include storing data comparing the pressure sensor operation time with the button operation time, and storing the volume and density of virtual smoke determined based on the compared data in the learned smoking pattern.

[0144] According to an embodiment, the volume and density of the virtual smoke may vary depending on the smoking pattern of the user. For example, if the user's exhalation time is shorter than the user's puff time, the volume of the virtual smoke may be increased to produce thicker smoke. If the user's puff time sensed by the pressure sensor is 2 seconds, but the user presses and releases the button for 1.5 seconds, the volume and density of the virtual smoke may be determined to produce more smoke and thicker smoke. That is, since the user's puff time is the smoke inhalation time and the user's button actuation time is the exhalation time, if the user's exhalation time is shorter than the user's smoke inhalation time, the user exhales an equal amount of smoke in a short period of time, and therefore the electronic device 110 should produce a larger amount of smoke and thicker smoke in a short period of time.

[0145] <Camera data and pressure sensor data> According to an embodiment, a pressure sensor may be included in the aerosol generating device 500. The pressure sensor may be located inside or outside the second end 530. The pressure sensor may measure the pressure generated by the user's lips to detect inhalation.

[0146] 1 and 5, when a user according to an embodiment starts smoking, the electronic device 110 receives camera data from a camera (not shown) that detects the user's hand movement. The pressure sensor can obtain pressure sensor data when the user bites the second end 530 with his / her mouth to smoke. The electronic device 110 can predict the time of exhalation based on the camera data and pressure sensor data received via the dongle 130, i.e., the detection information.

[0147] The electronic device 110 according to an embodiment may be installed with an application capable of monitoring the user's hand movements, and thus may detect the user's smoking behavior data through a camera capable of monitoring the user's hand movements. The camera may monitor a situation in which the user brings the aerosol generating device 500 to the mouth to inhale smoke, inhales the smoke into the lungs by inhaling, and then separates the aerosol generating device 500 from the mouth to exhale the smoke. The data received from the camera and the application installed in the electronic device 110 may analyze the user's hand movements as described above to determine the amount of inhalation and the time of exhalation, etc.

[0148] According to an embodiment, the electronic device 110, the camera, and the aerosol generating device 500 operate as follows. The electronic device 110 in which the smoking device application is installed and the aerosol generating device 500 are connected using the dongle 130. When a user brings the aerosol generating device 500 to his / her mouth for smoking, the application of the electronic device 110 can analyze the user's hand movement using an image (e.g., camera data) captured by the camera. If it is determined through the camera image that the aerosol generating device 500 has reached the mouth and the pressure sensor operates (e.g., the measured pressure value is equal to or greater than a threshold), it can be determined that the user has started smoking. For example, the aerosol generating device 500 can transmit a pressure value measured by the pressure sensor to the electronic device 110 via the dongle 130. After the user starts smoking, the user's action of inhaling smoke into the lungs by inhaling, i.e., the action of separating the aerosol generating device 500 from the mouth, can be monitored using the camera image. By analyzing the interval between the start of smoking and the start of inhalation, the inhalation volume and the amount of inhalation can be estimated, and the time of exhalation can be determined based on the estimated inhalation volume and the amount of inhalation.

[0149] According to one embodiment, the operation of determining the expiration time based on the monitoring result, i.e., the detection information, includes an operation of determining the expiration time based on the interval between the first time and the second time. Hereinafter, the process of determining the expiration time is the same as that using the capacitance sensor, and therefore a description thereof will be omitted.

[0150] FIG. 6 is a flowchart of a virtual smoke image output method according to an embodiment.

[0151] According to one embodiment, operation 430 described above with reference to FIG. 4 may include the following operations 610 and 620.

[0152] At operation 610, the electronic device 110 may determine weather information based on a scene in the target video content or location information related to the target video content. For example, if the target video content is any video content, the weather information may be determined based on a scene in the target video content. As another example, if the target video content is real-time video content, the weather information may be determined based on location information related to the target video content.

[0153] In one embodiment, the electronic device 110 may determine weather information based on a scene in the target video content. For example, if the saturation and brightness of the scene in the target video content are above a certain level, the weather information may be determined to be daytime. For example, if an object (e.g., a flag, a person's hair, etc.) in the scene in the target video content is swaying to the left, the wind direction in the scene may be determined to be leftward, and the wind speed may also be determined depending on the degree of swaying. For example, if the brightness and saturation of the scene in the target video content are low and it is raining, the amount of precipitation and humidity may be determined accordingly.

[0154] In one embodiment, the electronic device 110 may determine weather information based on location information related to the target video content. The weather information may be provided through an open API service based on the location information related to the target video content, such as a weather forecast from the Korea Meteorological Administration. For example, the location information related to the target video content may be Yeoksam 1 Building, Gangnam-gu, and the weather information may include temperature (e.g., 28.9°C), humidity (e.g., 84%), sunset time (e.g., 19:58), and wind direction (e.g., southwesterly).

[0155] In operation 620, the electronic device 110 may generate a virtual smoke image based on weather information and detection information. For example, the weather information corresponding to the target image content may include one or more of wind speed, wind direction, precipitation, temperature, humidity, fine dust, ultraviolet light, sunset time, and day / night. For example, the detection information regarding the state of the aerosol generating device 120 may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a body part of a user), detection value information of the capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of the camera of the aerosol generating device 120, smoking patterns learned using the pressure sensor and button of the aerosol generating device 120, and camera data and pressure sensor data.

[0156] In one embodiment, operation 620 may include determining attributes of the virtual smoke based on the weather information and the detection information, and generating a virtual smoke video based on the attributes of the virtual smoke. For example, the attributes of the virtual smoke may include one or more of the size, shape, duration, volume, density, and color of the virtual smoke, without being limited herein.

[0157] According to one example, a virtual smoke image may be generated such that the virtual smoke is scattered according to the wind speed and wind direction of the weather information. According to one example, a virtual smoke image may be generated such that the size (or smoke volume), concentration, volume, or color of the smoke appears according to the amount of precipitation, temperature, humidity, ultraviolet light, and fine dust of the weather information, and the smoking pattern of the user of the detection information. For example, a virtual smoke image may be generated such that the smoke becomes denser and darker in color as the humidity is higher and the user's exhalation time is shorter than the smoke inhalation time. As another example, a virtual smoke image may be generated such that the smoke disappears faster as the wind speed is higher.

[0158] FIG. 7 is a flowchart of a virtual smoke image output method according to an embodiment.

[0159] According to one embodiment, operation 430 described above with reference to FIG. 4 may include the following operations 710 and 720.

[0160] In operation 710, the electronic device 110 may determine attributes of the virtual smoke based on the weather information and the detection information.

[0161] In one embodiment, the size, duration, volume, and concentration of the virtual smoke may be determined to be proportional to the smoke inhalation time or the inhalation time based on the smoke inhalation time predicted using one or more of the detection information regarding the state of the aerosol generating device 120. For example, if the inhalation time is long, the duration of the virtual smoke may be determined to be the same or proportionally longer.

[0162] In one embodiment, the shape of the virtual smoke may be determined based on wind speed and camera monitoring information. When the camera of the electronic device 110 receives information monitoring the movement of an object, for example, the user's lips, the camera monitoring information may include information about the pattern of the lips. For example, if the user's lips are pursed, the shape of the virtual smoke may be determined to be thin and long. As another example, if the wind speed is high, it may be determined that the virtual smoke flies away quickly without maintaining a specific shape.

[0163] In one embodiment, the volume, density, and color of the virtual smoke may be determined according to precipitation, temperature, humidity, ultraviolet light, fine dust, whether it is day / night, and the user's smoking pattern. For example, if the user's exhalation time is shorter than the user's smoke inhalation time, it may be determined that the volume of the virtual smoke is increased to produce thicker smoke, and the color of the virtual smoke may be determined to be darker. As another example, at night, the color of the virtual smoke may be determined to be lighter so that the smoke is more visible. As another example, it may be determined that the higher the precipitation and humidity, the thicker the volume or density of the virtual smoke.

[0164] In one embodiment, the electronic device 110 can directly receive user input regarding attributes of the virtual smoke. For example, the color of the virtual smoke can be various colors such as blue, purple, etc., depending on the user input, as well as a general neutral color. As another example, the shape of the virtual smoke can be a donut shape.

[0165] In one embodiment, the detection information regarding the state of the aerosol generating device 120 is pressure sensor data, and may include airflow detection information due to the user's inhalation. When the internal flow air pressure where the pressure sensor of the aerosol generating device 120 is installed is high, the virtual smoke may be determined to be large and long lasting even if the inhalation time is short.

[0166] At operation 720, the electronic device 110 may generate a video of the virtual smoke based on the attributes of the virtual smoke.

[0167] In one embodiment, the virtual smoke image may be generated by further adjusting the pre-defined basic attributes based on the attributes of the virtual smoke determined in operation 710 .

[0168] FIG. 8 is a flowchart of a virtual smoke image output method according to an embodiment.

[0169] According to one embodiment, operation 440 described above with reference to FIG. 4 may include the following operations 810 and 820.

[0170] In operation 810, the electronic device 110 determines an exhalation time when the user of the aerosol generating device 120 starts exhaling based on the detection information regarding the state of the aerosol generating device 120. The detection information regarding the state of the aerosol generating device 120 may include one or more of distance information between the aerosol generating device 120 and an object (e.g., a body part of the user), detection value information of the capacitance sensor of the aerosol generating device 120, orientation change information of the aerosol generating device 120, monitoring information of the camera of the aerosol generating device 120, smoking patterns learned using the pressure sensor and button of the aerosol generating device 120, camera data and pressure sensor data, and button input information.

[0171] At operation 820, the electronic device 110 outputs the virtual smoke image at the time of exhalation. In one embodiment, the virtual smoke image is output via a display of the electronic device 110.

[0172] In one embodiment, when the target video content is a content that is pre-created to include weather information corresponding to a scene in the target video content, i.e., a video content dedicated to smoking, the virtual smoke video may be output so as to be combined with the scene in the target video content. For example, the video may be output so that virtual smoke appears in the scene in the target video content.

[0173] In one embodiment, when the target video content is not a smoking-specific video content but any video content or real-time video content, the virtual smoke video may be output by being superimposed on the target video content being output by the electronic device 110. For example, the electronic device 110 may output an image by superimposing the image using a layer for outputting the target video content and a virtual smoke video layer. The user may observe the superimposed image.

[0174] Even if a user cannot visually observe smoke in reality, the user can observe smoke through an image in which virtual smoke appears, and can satisfy the sensation of smoking through the observed smoke.

[0175] The above-described embodiments may be implemented with hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications executed on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, although it has been described that one processing device is used, a person skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0176] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure or instruct the processing device to operate as desired, either individually or collectively. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may also be distributed over network-coupled computer systems, and stored or executed in a distributed manner. The software and data may be stored in a computer-readable recording medium.

[0177] The method according to the embodiment may be realized in the form of program instructions executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, and the like, alone or in combination, and the program instructions recorded on the medium may be those specially designed or constructed for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

[0178] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of an embodiment, and vice versa.

[0179] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations thereto. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

[0180] Therefore, other embodiments, other examples, and equivalents to the claims are within the scope of the following claims.

Claims

1. A method for outputting a virtual smoke image by an electronic device, comprising: outputting the target video content selected by the user via a display of the electronic device; receiving detection information from an aerosol generating device coupled to the electronic device, the detection information relating to a state of the aerosol generating device; generating a virtual smoke image based on the weather information and the detection information corresponding to the target image content; outputting the virtual smoke image via the display of the electronic device; Including, the weather information is included within the target video content or is determined based on location information related to the target video content; A method for outputting virtual smoke images.

2. The target video content is a content prepared in advance to include the weather information corresponding to a scene in the target video content, The meteorological information is stored in advance as a database for each of one or more items including at least one of wind speed, wind direction, precipitation, temperature, or humidity, or a combination thereof; The virtual smoke image output method according to claim 1 .

3. The virtual smoke image is output to be composited with the scene of the target image content. The virtual smoke image output method according to claim 2 .

4. The weather information is further determined based on a scene of the target video content, and the operation of generating a virtual smoke image based on the weather information corresponding to the target video content and the detection information includes: determining the weather information based on at least one of saturation, brightness, objects, or weather of the scene in the target video content, or a combination thereof; and generating the virtual smoke image based on the meteorological information and the detection information; Including, The virtual smoke image output method according to claim 1 .

5. The operation of generating a virtual smoke image based on the weather information and the detection information corresponding to the target image content includes: determining the weather information based on location information related to the target video content via an open application programming interface (API), the target video content being real-time video content; and generating the virtual smoke image based on the meteorological information and the detection information; Including, The virtual smoke image output method according to claim 1 .

6. The operation of outputting the virtual smoke image via a display of the electronic device includes: and outputting the virtual smoke image by superimposing the virtual smoke image on the target image content being output by the electronic device. Including, The virtual smoke image output method according to claim 4.

7. The operation of outputting the virtual smoke image via a display of the electronic device includes: and outputting the virtual smoke image by superimposing the virtual smoke image on the target image content being output by the electronic device. Including, The virtual smoke image output method according to claim 5.

8. The operation of generating a virtual smoke image based on the weather information and the detection information corresponding to the target image content includes: determining attributes of the virtual smoke based on the weather information and the detection information, the attributes of the virtual smoke including at least one of a size, a shape, and a duration of the virtual smoke; and generating the virtual smoke image based on the attributes of the virtual smoke; Including, The virtual smoke image output method according to claim 1 .

9. The operation of outputting the virtual smoke image via the display of the electronic device includes: determining an exhalation time when a user of the aerosol generating device starts exhaling based on the detection information; and outputting the virtual smoke image at the time of exhalation; Including, The virtual smoke image output method according to claim 1 .

10. The operation of determining an exhalation time when a user of the aerosol generating device starts exhaling based on the detection information includes: determining the exhalation time based on a first time when a distance between one side of the aerosol generating device and the body part of the user reaches within a first threshold value, and a second time when the distance between the one side of the aerosol generating device and the body part of the user exceeds a second threshold value after the first time; Including, The virtual smoke image output method according to claim 9.

11. The operation of determining an exhalation time when a user of the aerosol generating device starts exhaling based on the detection information includes: determining the exhalation time based on a change in capacitance of the aerosol generating device; Including, The virtual smoke image output method according to claim 9.

12. The operation of determining the exhalation time based on the change in capacitance of the aerosol generating device includes: determining the exhalation time based on a first time at which a value of the capacitance of the aerosol generating device, as sensed by the aerosol generating device, falls within a first threshold value and a second time after the first time at which the value of the capacitance of the aerosol generating device exceeds a second threshold value; Including, The virtual smoke image output method according to claim 11.

13. The method of claim 1 , wherein the connection request is received from the aerosol generating device that has established a wireless communication channel with a dongle connected to the electronic device via the dongle.

14. The electronic device and the aerosol generating device directly establish a wireless communication channel; the detection information is received from the aerosol generating device via the wireless communication channel; The virtual smoke image output method according to claim 1 .

15. A program for causing a processor of the electronic device to execute the virtual smoke image output method described in any one of claims 1 to 14.

16. 1. An electronic device comprising: A display and A memory in which a program for outputting a virtual smoke image is recorded; A processor for executing the program; Including, The processor, outputting a target video content selected by a user via the display; receiving detection information from an aerosol generating device coupled to the electronic device, the detection information relating to a state of the aerosol generating device; generating a virtual smoke image based on the weather information and the detection information corresponding to the target image content; outputting the virtual smoke image via the display of the electronic device; Do the following: the weather information is included within the target video content or is determined based on location information related to the target video content; electronic equipment.

Images