Electronic device processing touch input data and controlling method thereof

Abstract

According to various embodiments, the electronic device includes a touch sensor IC configured to acquire touch input data through an application processor and a touch sensor, wherein the application processor acquires first touch input data from the touch sensor IC and determines whether to perform a preset function based on the first touch input data, acquires second touch input data from the touch sensor IC and determines whether to perform the preset function based on the second touch input data, determines whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data, and is configured to transmit first information regarding the second touch input data to the touch sensor IC in response to the determination that the second touch input data indicates a stable state, and the touch sensor IC may be configured to acquire third touch input data and determine whether to perform the preset function based on the first information and the third touch input data. Various other embodiments are possible.

Description

Technology Field

[0001] Various embodiments relate to an electronic device for processing touch input data and a method for controlling an electronic device for processing touch input data. Background Technology

[0002] Electronic devices capable of performing various functions by processing touch input data are widely available. Referring to FIG. 1, an example of pixel values ​​(100) representing touch input data obtained through a touch sensor is illustrated when a single touch input is made using a finger at a specific location on the touch screen of the electronic device. The pixel values ​​(100) may correspond to the capacitance at the location of each pixel of the touch sensor of the electronic device. The touch sensor may transmit the pixel values ​​(100) as touch input data to a touch sensor IC. Among the pixel values ​​(100), it can be seen that the pixel values ​​of the pixels (101, 102, 103, 104, 105, 106, 107, 108) surrounding the location where the touch input occurred are higher than the pixel values ​​of other pixels.

[0003] The touch sensor IC transmits all pixel values ​​(100) as touch input data to the application processor, and the application processor can analyze all pixel values ​​(100) represented by the touch input data. Using the touch input data, the application processor can not only calculate the coordinates where the touch input occurred, but also determine a force key input based on the intensity of the touch input, distinguish between a situation where the user simply holds the electronic device without the intention to make a touch input and a situation where the user intends to make a touch input, or determine a situation where the user's body part is close to the electronic device to make a phone call. The application processor can generate a classification model using machine learning and can analyze the touch input data using the generated classification model. The application processor can perform various functions based on the analysis results of the touch input data. The problem to be solved

[0004] When analyzing touch input data using a classification model generated by machine learning in an application processor, the application processor may be unable to perform the analysis of touch input data when it enters a sleep state as the display is turned off. If the application processor is kept in a wake-up state to ensure that touch input data can always be analyzed, power consumption may increase as the application processor continues to operate. Additionally, since changes in display brightness when the display is turned off or on cause noise in the touch input data, the accuracy of the analysis results may be limited because the touch input data cannot be continuously analyzed before and after the display turns off or on.

[0005] An electronic device according to one embodiment can analyze touch input data at a touch sensor IC without using machine learning in an application processor when the amount of change over time of touch input data is small, that is, when the touch input data indicates a stable state. means of solving the problem

[0006] An electronic device according to one embodiment includes a touch sensor IC configured to acquire touch input data through an application processor and a touch sensor, wherein the application processor acquires first touch input data from the touch sensor IC and determines whether to perform a preset function based on the first touch input data, acquires second touch input data from the touch sensor IC and determines whether to perform the preset function based on the second touch input data, determines whether to determine whether to perform the preset function based on the first touch input data and the second touch input data, determines whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data, and in response to the determination that the second touch input data indicates a stable state, transmits first information regarding the second touch input data to the touch sensor IC, and the touch sensor IC may be configured to acquire third touch input data and determine whether to perform the preset function based on the first information and the third touch input data.

[0007] A method performed in an electronic device comprising an application processor and a touch sensor IC according to one embodiment may include, in the application processor, an operation of acquiring a first touch input data from the touch sensor IC; in the application processor, an operation of determining whether to perform a preset function based on the first touch input data; in the application processor, an operation of acquiring a second touch input data from the touch sensor IC; in the application processor, an operation of determining whether to perform the preset function based on the second touch input data; in the application processor, an operation of determining whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data; in the application processor, an operation of transmitting first information regarding the second touch input data to the touch sensor IC in response to the determination that the second touch input data indicates a stable state; in the touch sensor IC, an operation of acquiring a third touch input data; and in the touch sensor IC, an operation of determining whether to perform the preset function based on the first information and the third touch input data. Effects of the invention

[0008] According to one embodiment, an electronic device for processing touch input data and a method for controlling the electronic device for processing touch input data may be provided. The electronic device according to one embodiment analyzes touch input data using a touch sensor IC instead of an application processor when the touch input data indicates a stable state, so that touch input data can be analyzed even when the application processor has entered a sleep state, and the amount of power consumed can be reduced.

[0009] In addition, the electronic device according to one embodiment does not immediately switch the application processor to a sleep state when the display is turned off, but maintains a wake-up state and switches to a sleep state when the touch input data indicates a stable state, and switches the application processor to a wake-up state before the display is turned on when the touch sensor IC determines that the touch input data indicates an unstable state, so that the touch input data before and after the display is turned off or on can be continuously analyzed, thereby enabling analysis with high accuracy. Brief explanation of the drawing

[0010] Figure 1 illustrates an example of touch input data. FIG. 2 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 3 is a block diagram of a display device according to various embodiments. FIG. 4 illustrates operations performed in an electronic device according to various embodiments. FIGS. 5a to 5c illustrate examples of touch input data according to various embodiments. FIG. 6 illustrates examples of touch input data according to various embodiments. FIGS. 7a and 7b illustrate examples of touch input data according to various embodiments. FIG. 8 illustrates operations performed in an electronic device according to various embodiments. FIG. 9 illustrates the operation of an application processor and a touch sensor IC in an electronic device according to various embodiments. FIG. 10 illustrates operations performed in an electronic device according to various embodiments. FIG. 11 illustrates the operation of an application processor and a touch sensor IC in an electronic device according to various embodiments. Specific details for implementing the invention

[0011] FIG. 2 is a block diagram of an electronic device (201) in a network environment (200) according to various embodiments. Referring to FIG. 2, in the network environment (200), the electronic device (201) may communicate with an electronic device (202) through a first network (298) (e.g., a short-range wireless communication network) or with an electronic device (204) or a server (208) through a second network (299) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (201) may communicate with the electronic device (204) through a server (208). According to one embodiment, the electronic device (201) may include a processor (220), memory (230), input module (250), sound output module (255), display module (260), audio module (270), sensor module (276), interface (277), connection terminal (278), haptic module (279), camera module (280), power management module (288), battery (289), communication module (290), subscriber identification module (296), or antenna module (297). In some embodiments, at least one of these components (e.g., connection terminal (278)) may be omitted from the electronic device (201), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (276), camera module (280), or antenna module (297)) may be integrated into a single component (e.g., display module (260)).

[0012] The processor (220) can control at least one other component (e.g., a hardware or software component) of the electronic device (201) connected to the processor (220) by executing software (e.g., a program (240)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (220) can store commands or data received from other components (e.g., a sensor module (276) or a communication module (290)) in volatile memory (232), process the commands or data stored in volatile memory (232), and store the resulting data in non-volatile memory (234). According to one embodiment, the processor (220) may include a main processor (221) (e.g., a central processing unit or an application processor) or an auxiliary processor (223) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (201) includes a main processor (221) and an auxiliary processor (223), the auxiliary processor (223) may be configured to use less power than the main processor (221) or to be specialized for a designated function. The auxiliary processor (223) may be implemented separately from the main processor (221) or as part thereof.

[0013] The auxiliary processor (223) may control at least some of the functions or states associated with at least one component of the electronic device (201) (e.g., display module (260), sensor module (276), or communication module (290)) on behalf of the main processor (221) while the main processor (221) is in an inactive (e.g., sleep) state, or together with the main processor (221) while the main processor (221) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (223) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (280) or communication module (290)). According to one embodiment, the auxiliary processor (223) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (201) itself where the artificial intelligence is performed, or through a separate server (e.g., server (208)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0014] The memory (230) can store various data used by at least one component of the electronic device (201) (e.g., processor (220) or sensor module (276)). The data may include, for example, software (e.g., program (240)) and input or output data for related commands. The memory (230) may include volatile memory (232) or non-volatile memory (234).

[0015] The program (240) may be stored as software in memory (230) and may include, for example, an operating system (242), middleware (244), or an application (246).

[0016] The input module (250) can receive commands or data to be used for a component of the electronic device (201) (e.g., processor (220)) from outside the electronic device (201) (e.g., user). The input module (250) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0017] The sound output module (255) can output a sound signal to the outside of the electronic device (201). The sound output module (255) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0018] The display module (260) can visually provide information to an external (e.g., user) of the electronic device (201). The display module (260) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (260) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0019] The audio module (270) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (270) can acquire sound through the input module (250) or output sound through the sound output module (255) or an external electronic device (e.g., electronic device (202)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (201).

[0020] The sensor module (276) can detect the operating state of the electronic device (201) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (276) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0021] The interface (277) may support one or more specified protocols that can be used for the electronic device (201) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (202)). According to one embodiment, the interface (277) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0022] The connection terminal (278) may include a connector through which the electronic device (201) can be physically connected to an external electronic device (e.g., electronic device (202)). According to one embodiment, the connection terminal (278) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0023] The haptic module (279) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (279) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0024] The camera module (280) can capture still images and video. According to one embodiment, the camera module (280) may include one or more lenses, image sensors, image signal processors, or flashes.

[0025] The power management module (288) can manage the power supplied to the electronic device (201). According to one embodiment, the power management module (288) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0026] The battery (289) can supply power to at least one component of the electronic device (201). According to one embodiment, the battery (289) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0027] The communication module (290) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (201) and an external electronic device (e.g., electronic device (202), electronic device (204), or server (208)), and the performance of communication through the established communication channel. The communication module (290) may include one or more communication processors that operate independently of the processor (220) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (290) may include a wireless communication module (292) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (294) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (204) via a first network (298) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (299) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (292) can identify or authenticate the electronic device (201) within a communication network such as the first network (298) or the second network (299) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (296).

[0028] The wireless communication module (292) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (292) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (292) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (292) can support various requirements specified in the electronic device (201), external electronic device (e.g., electronic device (204)), or network system (e.g., second network (299)). According to one embodiment, the wireless communication module (292) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0029] An antenna module (297) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (297) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (297) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (298) or a second network (299), may be selected from the plurality of antennas, for example, by a communication module (290). A signal or power may be transmitted or received between the communication module (290) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (297).

[0030] According to various embodiments, the antenna module (297) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0031] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0032] According to one embodiment, commands or data may be transmitted or received between the electronic device (201) and an external electronic device (204) through a server (208) connected to a second network (299). Each of the external electronic devices (202, or 204) may be the same or different type of device as the electronic device (201). According to one embodiment, all or part of the operations performed on the electronic device (201) may be performed on one or more of the external electronic devices (202, 204, or 208). For example, if the electronic device (201) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (201) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (201). The electronic device (201) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (201) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (204) may include an Internet of Things (IoT) device. The server (208) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (204) or the server (208) may be included within the second network (299).The electronic device (201) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0033] FIG. 3 is a block diagram (300) of a display device (260) according to various embodiments. Referring to FIG. 3, the display device (260) may include a display (310) and a display driver IC (DDI) (330) for controlling the display. The DDI (330) may include an interface module (331), a memory (333) (e.g., a buffer memory), an image processing module (335), or a mapping module (337). The DDI (330) may receive image information, for example, image data or an image control signal corresponding to a command for controlling the image data, from another component of the electronic device 201 through the interface module (331). For example, according to one embodiment, image information may be received from a processor (220) (e.g., main processor (221) (e.g., application processor)) or an auxiliary processor (223) (e.g., graphics processing unit) that operates independently of the functions of the main processor (221). The DDI (330) may communicate with the touch circuit (350) or sensor module (276), etc., through the interface module (331). Additionally, the DDI (330) may store at least a portion of the received image information in memory (333), for example, in frame units. The image processing module (335) may perform preprocessing or postprocessing (e.g., resolution, brightness, or size adjustment) on at least a portion of the image data based at least on the characteristics of the image data or the characteristics of the display (310), for example. The mapping module (337) may generate voltage values ​​or current values ​​corresponding to the image data preprocessed or postprocessed through the image processing module (235). According to one embodiment, voltage values ​​or current The generation of values ​​can be performed, for example, based on at least some of the attributes of the pixels of the display (310) (e.g., array of pixels (RGB stripe or pentile structure), or size of each subpixel).At least some pixels of the display (310) are driven, for example, based on at least some of the voltage value or current value, so that visual information (e.g., text, image, or icon) corresponding to the image data can be displayed through the display (310).

[0034] According to one embodiment, the display device (260) may further include a touch circuit (350). The touch circuit (350) may include a touch sensor (351) and a touch sensor IC (353) for controlling the same. The touch sensor IC (353) may control the touch sensor (351) to detect a touch input or hovering input for a specific location on the display (310), for example. For example, the touch sensor IC (353) may detect a touch input or hovering input by measuring a change in a signal (e.g., voltage, light intensity, resistance, or charge) for a specific location on the display (310). The touch sensor IC (353) may provide information regarding the detected touch input or hovering input (e.g., location, area, pressure, or time) to the processor (220). According to one embodiment, at least a part of the touch circuit (350) (e.g., touch sensor IC (353)) may be included as part of the display driver IC (330) or the display (310), or as part of another component (e.g., auxiliary processor (223)) placed outside the display device (260).

[0035] According to one embodiment, the display device (260) may further include at least one sensor of the sensor module (276) (e.g., fingerprint sensor, iris sensor, pressure sensor, or light sensor) or a control circuit for the same. In this case, the at least one sensor or the control circuit for the same may be embedded in a part of the display device (260) (e.g., display (310) or DDI (330)) or a part of the touch circuit (350). For example, if the sensor module (276) embedded in the display device (260) includes a biometric sensor (e.g., fingerprint sensor), the biometric sensor may obtain biometric information (e.g., fingerprint image) associated with a touch input through a part of the display (310). As another example, if the sensor module (276) embedded in the display device (260) includes a pressure sensor, the pressure sensor may obtain pressure information associated with a touch input through a part or the entire area of ​​the display (310). According to one embodiment, a touch sensor (351) or a sensor module (276) may be placed between pixels of a pixel layer of a display (310), or on top of or below the pixel layer.

[0036] FIG. 4 illustrates an operation performed in an electronic device according to various embodiments. In operation 410, an application processor (e.g., main processor (221)) of an electronic device (e.g., electronic device (201)) may obtain first touch input data from a touch sensor IC (e.g., touch sensor IC (353)). According to various embodiments, the first touch input data may include data based on a change in capacitance corresponding to pixels included in the touch sensor (351), which is obtained through the touch sensor (351). According to various embodiments, the touch sensor IC (353) may identify the coordinates where a touch input occurred based on the data obtained through the touch sensor (351), and may include the identified coordinates in the first touch input data along with data corresponding to pixels included in the touch sensor (351) and transmit them to the application processor (221).

[0037] According to various embodiments, the touch sensor IC (353) can transmit first touch input data to the application processor (221) through an interface such as I2C (inter-integrated circuit) and / or SPI (serial peripheral interface).

[0038] In operation 420, the application processor (221) can determine whether to perform a preset function based on the first touch input data. According to various embodiments, the application processor (221) can generate a classification model using machine learning and apply the classification model to the first touch input data to determine a force key input, determine a situation where the user simply grasps the electronic device without intending to make a touch input, or determine a situation where a part of the user's body is close to the electronic device. According to various embodiments, if the first touch input data does not indicate information about the coordinates where the touch input occurred, the application processor (221) can determine the coordinates where the touch input occurred based on the first touch input data.

[0039] In operation 420, the application processor (221) can perform a function corresponding to the determined situation and / or input based on the coordinates of a force key input, a situation where the user is holding the electronic device, a situation where a part of the user's body is close to the electronic device, and / or a touch input.

[0040] According to various embodiments, when the application processor (221) performs the 420 operation, it may drive the digital signal processor (DSP) and the graphics processing unit (GPU) included in the application processor (221) to use a classification model using machine learning. Accordingly, the application processor (221) may consume relatively high power when performing the 420 operation.

[0041] In operation 430, the application processor (221) of the electronic device (201) can acquire second touch input data from the touch sensor IC (353). Depending on various embodiments, the second touch input data may be data acquired later than the first touch input data. The details described above with reference to operation 410 with respect to the first touch input data may be applied equally to the second touch input data.

[0042] In operation 440, the application processor (221) can determine whether to perform a preset function based on the second touch input data. Details regarding operation 420 may be similarly applied to operation 440.

[0043] In operation 450, the application processor (221) can determine whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data. Depending on various embodiments, the second touch input data indicating a stable state may mean that the difference between the first touch input data and the second touch input data is not large.

[0044] If it is determined that the second touch input data does not indicate a stable state in operation 450, the electronic device (201) may return to operation 410 to obtain new touch input data. According to various embodiments, the electronic device (201) may omit operations 410 and 420, treat the existing second touch input data as new first touch input data, and obtain new second touch input data in operation 430 to perform operations 440 and 450.

[0045] According to various embodiments, the application processor (221) can confirm that the second touch input data indicates a stable state when the difference between the pixel values ​​represented by the first touch input data and the pixel values ​​represented by the second touch input data is below a certain level. Refer to FIGS. 5a through 5c to explain examples of specific criteria for confirming a stable state based on pixel values. FIG. 5a illustrates pixel values ​​represented by touch input data acquired when the screen is turned on. FIG. 5b illustrates pixel values ​​represented by touch input data acquired immediately after the screen is turned on and then turned off. FIG. 5c illustrates pixel values ​​represented by touch input data acquired after a certain amount of time has elapsed since the screen was turned on and then turned off.

[0046] According to various embodiments, the application processor (221) can determine whether the second touch input data indicates a stable state by comparing the pixel values ​​of corresponding locations represented by the first touch input data and the second touch input data. For example, when the first touch input data represents pixel values ​​(500a) shown in FIG. 5a and the second touch input data represents pixel values ​​(500b) shown in FIG. 5b, the application processor (221) checks the difference between the pixel value of the pixels (e.g., pixel (501a)) included in the pixel values ​​(500a) shown in FIG. 5a and the pixel value of the positionally corresponding pixels (e.g., pixel (501b)) among the pixel values ​​(500b) shown in FIG. 5b, checks the maximum and minimum values ​​among the differences equal to the number of pixels (180 in the example of FIG. 5a and FIG. 5b), and if the absolute values ​​of the maximum and minimum values ​​are each less than a specific value (e.g., 30), the second touch input data can be confirmed to be in a stable state. In the example of FIG. 5a and FIG. 5b, the maximum difference between the pixel values ​​of the corresponding positions is 101 and the minimum difference is 67, so the application processor (221) can confirm that the second touch input data corresponding to the pixel values ​​(500b) shown in FIG. 5b does not indicate a stable state.

[0047] For example, when the first touch input data represents the pixel values ​​(500b) shown in FIG. 5b and the second touch input data represents the pixel values ​​(500c) shown in FIG. 5c, the maximum difference between the pixel values ​​at the corresponding positions is -10 and the minimum difference is -21. Since the absolute values ​​of the maximum and minimum values ​​are less than a specific value (e.g., 30), the application processor (221) can confirm that the second touch input data represents a stable state when the touch input data shown in FIG. 5c is the second touch input data, that is, after a certain amount of time has passed since the screen was turned on and then turned off.

[0048] According to various embodiments, the application processor (221) checks the average of pixel values ​​included in the rows and columns of pixels for the first touch input data and the second touch input data, respectively, and checks the difference in the average of pixel values ​​for the rows and columns of the corresponding positions, and if the absolute value of the maximum difference is less than a specific value (e.g., 15), it can check that the second touch input data indicates a stable state. For example, when the first touch input data represents the pixel values ​​(500a) shown in FIG. 5a and the second touch input data represents the pixel values ​​(500b) shown in FIG. 5b, the application processor (221) can check the average of pixel values ​​included in column rx0 among the pixel values ​​(500a), check the average of pixel values ​​for columns rx1 through rx9 respectively, and check the average of pixel values ​​for rows tx0 through tx17 respectively. Additionally, the application processor (221) can check the average of pixel values ​​for columns rx0 to rx9 and the average of pixel values ​​for rows tx0 to tx17 for pixel values ​​(500b). Then, the application processor (221) can check the difference between the average of pixel values ​​in column rx0 of pixel values ​​(500a) and the average of pixel values ​​in column rx0 of pixel values ​​(500b), and can check the difference between the averages of pixel values ​​corresponding to the first touch input data and the second touch input data for columns rx1 to rx9 and rows tx0 to tx17, respectively. In the example of FIGS. 5A and 5B, among columns rx0 to rx9 and rows tx0 to tx17, the difference in the average pixel value of column rx8 is the largest at 95.2, and the absolute value of this value exceeds a preset value of 15. Therefore, the application processor (221) can confirm that the second touch input data does not indicate a stable state.

[0049] For example, when the first touch input data represents the pixel values ​​(500b) shown in FIG. 5b and the second touch input data represents the pixel values ​​(500c) shown in FIG. 5c, among columns rx0 to rx9 and rows tx0 to tx17, the difference in the average pixel value of row tx0 is the largest at -4.2, and the absolute value of this value, 4.2, is smaller than the preset value of 15. Therefore, the application processor (221) can confirm that the second touch input data represents a stable state.

[0050] According to various embodiments, the application processor (221) can confirm that the second touch input data indicates a stable state when the difference between the characteristics of the object represented by the first touch input data and the characteristics of the object represented by the second touch input data is below a certain level. Refer to FIG. 6 to explain an example of a specific criterion for confirming a stable state based on the characteristics of the object. FIG. 6 illustrates an example of touch input data when a touch input is made using a finger on a touch screen of an electronic device. The pixel values ​​represented by the pixels (100) in FIG. 6 are the same as those in FIG. 1. According to various embodiments, the application processor (221) can define the area of ​​pixels (101, 102, 103, 104, 105, 106, 107, 108) whose pixel values ​​are higher than the pixel values ​​of other pixels as an object (610). In the example of FIG. 6, the application processor (221) defined an elliptical object (610). According to various embodiments, the application processor (221) can identify at least one of the center position of the elliptical object (610), the length of the major axis (611), or the length of the minor axis (612) as a characteristic of the object.

[0051] According to various embodiments, the application processor (221) can confirm that the second touch input data indicates a stable state when the difference between the result of classifying the first touch input data by a classification model and the result of classifying the second touch input data by a classification model is below a certain level. According to various embodiments, the classification model may be a classification model generated using machine learning, as described above in operation 420. Refer to FIGS. 7a and 7b to explain an example of a specific criterion for confirming a stable state based on the result of classifying the touch input data by a classification model. FIGS. 7a and 7b illustrate an example of touch input data when the user's ear of the electronic device is close to the touch screen. According to various embodiments, when the first touch input data represents the pixel values ​​(700a) shown in FIG. 7a and the second touch input data represents the pixel values ​​(700b) shown in FIG. 7b, the application processor (221) can identify the first object (710a) by classifying the first touch input data into a classification model, and can identify the location of the first object (710a), that the first object (710a) corresponds to ear proximity, the size of the first object (710a), and the proximity between the ear corresponding to the first object (710a) and the touch sensor (e.g., touch sensor (351)), and / or the second object (720a) associated with the first object (710a) as characteristics of the first object (710a). Additionally, the application processor (221) can identify the first object (710b) by classifying the second touch input data into a classification model, and can identify the location of the first object (710b), that the first object (710b) corresponds to ear proximity, the size of the first object (710b), and the proximity between the ear corresponding to the first object (710b) and the touch sensor (e.g., touch sensor (351)), and / or the second object (720b) ​​associated with the first object (710b) as characteristics of the first object (710b).The application processor (221) can confirm that the second touch input data is in a stable state when the difference between the characteristics of the first object (710a) and the characteristics of the first object (710b) is less than a certain level.

[0052] Although FIG. 4 illustrates a stable state being checked based on two touch input data, such as a first touch input data and a second touch input data, according to various embodiments, the application processor (221) may check a stable state based on three or more touch input data.

[0053] In operation 450, if it is confirmed that the second touch input data indicates a stable state, the application processor (221) may transmit first information regarding the second touch input data to the touch sensor IC (353) in operation 460. Depending on various embodiments, the first information may include the result of applying a classification model to the second touch input data. Depending on various embodiments, the first information may include the result of classifying the second touch input data and / or a feature map regarding the second touch input data. Depending on various embodiments, the application processor (221) may inform the touch sensor IC (353) through the first information that one or more objects represented by the second touch input data are not valid inputs that cause a preset function to be performed, but are objects that the touch sensor IC (353) should ignore. Depending on various embodiments, the first information may further include information regarding the state of the electronic device (201) that may introduce noise to the touch input data, such as the state of the display, temperature conditions, the charging state of the electronic device (201), and whether a call is in progress. According to various embodiments, the first information may further include data obtained through an accelerometer, a gyroscope, an illuminance sensor, and / or a proximity sensor included in the electronic device (201).

[0054] In operation 470, the touch sensor IC (353) can acquire third touch input data through the touch sensor (351). In operation 480, the touch sensor IC (353) can determine whether to perform a preset function based on the first information and the third touch input data. In various embodiments, the touch sensor IC (353) can determine whether to perform a preset function on its own without transmitting the third touch input data to the application processor (221). In various embodiments, the touch sensor IC (353) can determine whether to perform a preset function without using machine learning, in a way that has less data computation than using machine learning.

[0055] According to various embodiments, the application processor (221) may be switched to a sleep state after performing operation 460. According to various embodiments, the application processor (221) may be kept in a wake-up state while operations 470 and 480 are being performed. According to various embodiments, the application processor (221) may be kept in a wake-up state when the remaining amount of the battery (289) of the electronic device (201) falls below a specific first level or when the electronic device (201) enters a power saving mode. According to various embodiments, the application processor (221) may be switched to a sleep state when the remaining amount of the battery (289) of the electronic device (201) falls below a second level lower than the first level or when the electronic device (201) enters an ultra-low power saving mode.

[0056] When the application processor (221) maintains a wake-up state while the 470 operation and 480 operation are being performed, the application processor (221) maintains a wake-up state, does not receive third touch input data, does not perform classification operations based on machine learning, and checks whether to perform a preset function in a relatively simple manner in the touch sensor IC (353), so the power consumed by the electronic device (201) when the 470 operation and 480 operation are being performed may be less than when the 420 operation or 440 operation is being performed.

[0057] When the application processor (221) is switched to a sleep state after the 460 operation is performed, as the power consumption of the application processor (221) decreases, the power consumed by the electronic device (201) when the 470 operation and the 480 operation are performed may be less than when the 420 operation or the 440 operation is performed.

[0058] In operation 480, the touch sensor IC (353) uses the first information obtained from the application processor (221) to determine whether to perform a preset function based on the third touch input data, so it can have higher accuracy compared to relying only on the third touch input data.

[0059] FIG. 8 illustrates operations performed in an electronic device according to various embodiments. FIG. 9 illustrates the operations of an application processor and a touch sensor IC in an electronic device according to various embodiments. Specifically, 910 of FIG. 9 illustrates when the application processor (221) performs an operation, and 920 illustrates when the touch sensor IC (353) performs an operation.

[0060] In operation 810, the application processor (e.g., main processor (221)) of the electronic device (e.g., electronic device (201)) can determine whether to perform a preset function based on first touch input data obtained from a touch sensor IC (e.g., touch sensor IC (353)). Depending on various embodiments, the first touch input data in operation 810 may include touch input data obtained at different points in time. For operation 810, details of operations 410 through 440 of FIG. 4 may be applied. Referring to FIG. 9, the t1 interval corresponds to the interval in which operation 810 occurs. Within the t1 interval, it is illustrated that the touch sensor IC (353) obtains first touch input data through the touch sensor (351) and transmits the obtained first touch input data to the application processor (221). In FIG. 9, the first touch input data is transmitted from the touch sensor IC (353) to the application processor (221), as indicated by arrows (931, 932). The application processor (221) that has acquired the first touch input data checks whether to perform a preset function based on the acquired first touch input data, as indicated in the t1 interval of 910.

[0061] In operation 820, the application processor (221) can determine whether the first touch input data indicates a stable state. For operation 820, the details of operation 450 of FIG. 4 may be applied. If it is determined in operation 820 that the first touch input data does not indicate a stable state, the electronic device (201) may repeat operation 810.

[0062] If it is confirmed in operation 820 that the first touch input data indicates a stable state, the application processor (221) may transmit first information regarding the first touch input data to the touch sensor IC (353) in operation 830. The first information has been described above with reference to operation 460 of FIG. 4. Referring again to FIG. 9, the transmission of the first information by the application processor (221) to the touch sensor IC (353) is indicated by an arrow (941).

[0063] In operation 840, the touch sensor IC (353) acquires second touch input data and, based on the first information and the second touch input data, determines whether to perform a preset function. In operation 840, the second touch input data may include touch input data acquired at different points in time, and may be touch input data acquired at a later point in time than the first touch input data. For operation 840, the details of operations 470 and 480 of FIG. 4 may be applied.

[0064] According to various embodiments, the application processor (221) may enter a sleep state after performing the 830 operation, or may maintain a wake-up state even while the 840 operation is being performed. When the application processor (221) is switched to a sleep state after the 830 operation is performed, the power consumption of the application processor (221) is reduced, so the power consumed by the electronic device (201) when the 840 operation is performed may be less than when the 810 operation is performed. Even if the application processor (221) maintains a wake-up state while the 840 operation is being performed, the application processor (221) does not receive touch input data from the touch sensor IC (353), does not perform classification operations based on machine learning, and checks whether to perform a preset function in a relatively simple manner at the touch sensor IC (353), so the power consumed by the electronic device (201) when the 840 operation is performed may be less than when the 810 operation is performed.

[0065] In operation 850, the touch sensor IC (353) can determine whether the second touch input data indicates a stable state. Depending on various embodiments, the touch sensor IC (353) can determine whether the second touch input data indicates a stable state based on the difference between the pixel values ​​represented by the second touch input data and the touch input data acquired at a previous time point, or the difference between the characteristics of the object represented by the second touch input data and the touch input data acquired at a previous time point. Detailed methods for determining whether the second touch input data indicates a stable state, as described above with reference to FIGS. 5a through 5c and FIG. 6, may be applied in operation 850. Referring to FIG. 9, operation 840 and operation 850 may be performed in the t2 interval.

[0066] If it is confirmed that the second touch input data indicates a stable state in the 850 operation, the touch sensor IC (353) can continue to perform the 840 operation.

[0067] If it is determined in operation 850 that the second touch input data does not indicate a stable state, the touch sensor IC (353) may transmit the second touch input data to the application processor (221) in operation 860. According to various embodiments, in addition to the second touch input data in operation 860, the touch sensor IC (353) may transmit to the application processor (221) the results of image classification, image recognition, object detection, and / or image segmentation corresponding to the second touch input data determined by itself in operation 840. Referring to FIG. 9, the transmission of the second touch input data from the touch sensor IC (353) to the application processor (221) is indicated by an arrow (942).

[0068] After operation 860 is performed, the application processor (221) can return to operation 810 and determine whether to perform a preset function based on the received second touch input data. According to various embodiments, if the result of image classification, image recognition, object detection, and / or image segmentation corresponding to the second touch input data determined by the touch sensor IC (353) in operation 860 is transmitted to the application processor (221), the application processor (221) can transmit the result of image classification, image recognition, object detection, and / or image segmentation, in which a machine learning-based classification model is applied to the second touch input data in operation 810, to the touch sensor IC (353). After performing operation 810, the application processor (221) can repeat operation 820 and operation 810 until touch input data indicating a stable state is confirmed. Referring to FIG. 9, the touch sensor IC (353) transmitting touch input data to the application processor (221) during the t3 interval is indicated by arrows (951, 952).

[0069] In the t1 and t3 intervals, where the acquired touch input data does not indicate a stable state, the touch sensor IC (353) transmits the touch input data to the application processor (221), and the application processor (221) can analyze the touch input data using a classification model based on machine learning. On the other hand, after it is confirmed that the touch input data indicates a stable state in the t1 interval, the touch sensor IC (353) enters the t2 interval and analyzes the touch input data independently, and if it is confirmed that the touch input data does not indicate a stable state in the t2 interval, the touch input data can be analyzed again by the application processor (221) by entering the t3 interval.

[0070] According to various embodiments, the electronic device (201) can reduce power consumption by analyzing touch input data at the touch sensor IC (353) instead of the application processor (221) when the touch input data indicates a stable state. Although the touch sensor IC (353) has a smaller amount of computational processing capacity than the application processor (221), the accuracy of the touch input data analysis can be ensured because the touch sensor IC (353) analyzes the touch input data only when the touch input data indicates a stable state. Additionally, since the touch sensor IC (353) analyzes the touch input data using the first information received from the application processor (221), it can have higher accuracy compared to relying solely on the touch input data.

[0071] FIG. 10 illustrates operations performed in an electronic device according to various embodiments. FIG. 11 illustrates the operation of an application processor and a touch sensor IC in an electronic device according to various embodiments.

[0072] In operation 1010, the application processor (e.g., main processor (221)) of the electronic device (e.g., electronic device (201)) can determine whether to perform a preset function based on first touch input data obtained from the touch sensor IC (353) while the display (e.g., display module (260)) is turned on. Details regarding operations 410 to 440 of FIG. 4 may be applied to operation 1010.

[0073] Referring to FIG. 11, the interval prior to t1 may be the interval in which the 1010 operation is performed. Arrows (1111, 1112, 1113) may indicate that first touch input data is transmitted from the touch sensor IC (353) to the application processor (221). Arrow (1114) may indicate that, as it is confirmed to perform a preset function, the display content resulting from the application processor (221) performing the function is transmitted to the display (260).

[0074] In operation 1020, the application processor (221) checks whether the display off condition is satisfied, controls the display (260) to an off state based on the satisfaction of the display off condition, and determines whether to perform a preset function based on the first touch input data obtained from the touch sensor IC (353). The first touch input data obtained from the touch sensor IC (353) in operation 1020 may be touch input data obtained later than the first touch input data obtained in operation 1010. According to various embodiments, in operation 1020, even though the display (260) is controlled to an off state, the application processor (221) may maintain a wake-up state without entering a sleep state using a wakelock. Referring to FIG. 11, t1 may indicate the time when the display (260) is switched to an off state.

[0075] In operation 1030, the application processor (221) can check whether the first touch input data obtained in operation 1020 indicates a stable state. Details regarding operation 450 of FIG. 4 may be applied to operation 1030. If the first touch input data does not indicate a stable state, the application processor (221) may repeat operation 1020 until the first touch input data indicating a stable state is confirmed. Referring to FIG. 11, the interval between t1 and t2 may indicate the interval during which operation 1020 and operation 1030 are performed. Arrows (1121, 1122) may indicate that the first touch input data is transmitted from the touch sensor IC (353) to the application processor (221) in operation 1020.

[0076] If it is confirmed that the first touch input data obtained in operation 1020 indicates a stable state, the application processor (221) may transmit first information regarding the first touch input data to the touch sensor IC (353) in operation 1040. Details regarding operation 460 of FIG. 4 may be applied to operation 1040. Referring to FIG. 11, the arrow (1130) may indicate that the first information is transmitted from the application processor (221) to the touch sensor IC (353).

[0077] In operation 1050, the application processor (221) may enter a sleep state. Referring to FIG. 11, t2 may indicate the time when the application processor (221) enters a sleep state.

[0078] In operation 1060, the touch sensor IC (353) acquires second touch input data and, based on the first information and the second touch input data, determines whether to perform a preset function. In operation 1060, the second touch input data may include touch input data acquired at different points in time, and may be touch input data acquired at a later point in time than the first touch input data. For operation 1060, the details of operations 470 and 480 of FIG. 4 may apply. While operation 1060 is being performed, the application processor (221) is in a sleep state and the display (260) is controlled to be off, so the power consumed by the electronic device (201) may be lower than when operations 1010 through 1040 are being performed.

[0079] In operation 1070, the touch sensor IC (353) can determine whether the second touch input data indicates a stable state. For operation 1070, the details of operation 850 of FIG. 8 may be applied.

[0080] When it is confirmed that the second touch input data indicates a stable state, the touch sensor IC (353) returns to operation 1060 to acquire new touch input data, checks whether to perform a preset function based on the first information and the new touch input data, and checks whether the new touch input data indicates a stable state. Referring to FIG. 11, during the interval between t2 and t3, the touch sensor IC (353) can repeat the operation of acquiring new touch input data, checking whether to perform a preset function based on the first information and the new touch input data, and checking whether the new touch input data indicates a stable state.

[0081] If it is determined that the second touch input data does not indicate a stable state, the touch sensor IC (353) may transmit the second touch input data to the application processor (221) in operation 1080. Depending on various embodiments, the touch sensor IC (353) may transmit a wake-up signal to the application processor (221) along with the second touch input data to cause the application processor (221) to exit the sleep state, or transmit the wake-up signal to the application processor (221) before transmitting the second touch input data. For operation 1080, the details of operation 860 of FIG. 8 may apply. Referring to FIG. 11, the arrow (1140) may indicate that the touch sensor IC (353) transmits the second touch input data and the wake-up signal to the application processor (221). Depending on various embodiments, the touch sensor IC (353) may transmit an image analysis result performed independently based on the second touch input data to the application processor (221) along with the second touch input data in operation 1080. The image analysis results may include the results of image classification, image recognition, object detection, and / or image segmentation corresponding to the second touch input data. The arrow (1140) in FIG. 11 may indicate that the touch sensor IC (353) transmits the image analysis results it has performed itself to the application processor (221) in addition to the second touch input data and the wake-up signal.

[0082] In operation 1085, the application processor (221) can check whether the second touch input data satisfies the display-on condition. Depending on various embodiments, the application processor (221) can check whether the second touch input data satisfies the display-on condition by applying a machine learning-based classification model to the second touch input data. Depending on various embodiments, the application processor (221) can exit the sleep state and enter the wake-up state based on a wake-up signal, and perform operation 1085 in the wake-up state. Referring to FIG. 11, t3 may indicate the time when the application processor (221) enters the wake-up state.

[0083] According to various embodiments, when the application processor (221) receives an image analysis result performed by the touch sensor IC (353) based on the second touch input data from the touch sensor IC (353) in operation 1080, the application processor (221) may transmit the image analysis result obtained by applying a machine learning-based classification model to the second touch input data to the touch sensor IC (353). Referring to FIG. 11, the arrow (1150) may indicate that the application processor (221) transmits the image analysis result obtained by applying a machine learning-based classification model to the second touch input data to the touch sensor IC (353). According to various embodiments, the application processor (221) may not transmit the image analysis result obtained by applying a machine learning-based classification model to the second touch input data to the touch sensor IC (353). In other words, according to various embodiments, the arrow (1150) of FIG. 11 may be omitted.

[0084] If it is determined in operation 1085 that the second touch input data does not satisfy the display-on condition, the application processor (221) may perform operation 1040 again. The application processor (221) may transmit the result of applying a machine learning-based classification model to the second touch input data in operation 1040 to the touch sensor IC (353) as information regarding the second touch input data, and may enter a sleep state again in operation 1050.

[0085] If it is confirmed in operation 1085 that the second touch input data satisfies the display-on condition, the application processor (221) controls the display (260) to the ON state in operation 1090, obtains new touch input data again through the touch sensor IC (353) in operation 1010, and can determine whether to perform a preset function based on the new touch input data. According to various embodiments, even if it is confirmed that the second touch input data satisfies the display-on condition, the application processor (221) can transmit the result of applying a machine learning-based classification model to the second touch input data to the touch sensor IC (353). Referring to FIG. 11, t4 may indicate the time when the display (260) is switched to the ON state. The arrows (1161, 1162, 1163) may indicate that new touch input data obtained again through the touch sensor IC (353) in operation 1010 is transmitted to the application processor (221).

[0086] Referring to FIG. 11, an electronic device (201) according to various embodiments analyzes touch input data using machine learning in an application processor (221) at the time when the display (260) is switched from an on state to an off state or from an off state to an on state, so that noise resulting from the on / off switching of the display (260) becomes a learning target and can increase the accuracy of the touch input data analysis result.

[0087] In an electronic device (e.g., electronic device (201)) according to various embodiments, the device comprises an application processor (e.g., main processor (221)) and a touch sensor IC (e.g., touch sensor IC (353)) configured to acquire touch input data through a touch sensor, wherein the application processor (221) acquires first touch input data from the touch sensor IC (353), determines whether to perform a preset function based on the first touch input data, acquires second touch input data from the touch sensor IC (353), determines whether to perform the preset function based on the second touch input data, determines whether to determine whether to perform the preset function based on the first touch input data and the second touch input data, determines whether to determine whether to determine whether to determine whether to determine whether to determine whether to perform the preset function based on the first touch input data and the second touch input data, and, in response to the determination that the second touch input data indicates a stable state, transmits first information regarding the second touch input data to the touch sensor IC (353), and wherein the touch sensor IC (353) acquires third touch input data and determines whether to perform the preset function based on the first information and the third touch input data It can be configured to check.

[0088] According to various embodiments, the touch sensor IC (353) is configured to determine whether to perform the preset function based on the first information and the third touch input data, acquire the fourth touch input data, determine whether the fourth touch input data indicates a stable state based on the third touch input data and the fourth touch input data, and in response to the determination that the fourth touch input data indicates an unstable state, transmit the fourth touch input data to the application processor (221), and the application processor (221) may be configured to determine whether to perform the preset function based on the fourth touch input data.

[0089] According to various embodiments, the application processor (221) may be configured to determine whether the second touch input data indicates a stable state based on at least one of the difference between the pixel values ​​represented by the first touch input data and the pixel values ​​represented by the second touch input data, the difference between the characteristics of the object represented by the first touch input data and the characteristics of the object represented by the second touch input data, or the difference between the result of classifying the first touch input data by a classification model and the result of classifying the second touch input data by a classification model.

[0090] According to various embodiments, the application processor (221) may be configured to generate the classification model by machine learning.

[0091] According to various embodiments, the application processor (221) is configured to determine whether to perform the preset function by applying a classification model generated by machine learning to the first touch input data, and to determine whether to perform the preset function by applying the classification model to the second touch input data, and the first information may include the result of applying the classification model to the second touch input data.

[0092] According to various embodiments, the touch sensor IC (353) may be configured to determine whether the fourth touch input data indicates a stable state based on at least one of the difference between the pixel values ​​represented by the third touch input data and the pixel values ​​represented by the fourth touch input data, or the difference between the characteristics of the object represented by the third touch input data and the characteristics of the object represented by the fourth touch input data.

[0093] According to various embodiments, in claim 1, the application processor (221) may be configured to enter a sleep state after transmitting the first information to the touch sensor IC (353).

[0094] According to various embodiments, the electronic device further includes a display, and the application processor (221) may be configured to check that a display off condition is satisfied, and based on that the display off condition is satisfied, control the display to an off state, and acquire the first touch input data from the touch sensor IC (353) while the display is controlled to an off state.

[0095] According to various embodiments, the touch sensor IC (353) may be configured to determine whether to perform the preset function based on the first information and the third touch input data, acquire the fourth touch input data, determine whether the fourth touch input data indicates a stable state based on the third touch input data and the fourth touch input data, and in response to the determination that the fourth touch input data indicates an unstable state, transmit a wake-up signal and the fourth touch input data to the application processor (221) to cause the application processor (221) to exit the sleep state, and the application processor (221) may be configured to enter a wake-up state based on the wake-up signal and determine whether the display-on condition is satisfied based on the fourth touch input data.

[0096] According to various embodiments, the application processor (221) may be configured to apply a classification model generated by machine learning to the fourth touch input data to determine whether the display on condition is satisfied, and based on the determination that the display on condition is not satisfied: transmit the result of applying the classification model to the fourth touch input data to the touch sensor IC (353); enter the sleep state, and control the display to the on state based on the determination that the display on condition is satisfied.

[0097] A method performed in an electronic device (e.g., electronic device (201)) comprising an application processor (e.g., main processor (221)) and a touch sensor IC (e.g., touch sensor IC (353)) according to various embodiments, wherein the application processor (221) acquires first touch input data from the touch sensor IC (353); wherein the application processor (221) determines whether to perform a preset function based on the first touch input data; wherein the application processor (221) acquires second touch input data from the touch sensor IC (353); wherein the application processor (221) determines whether to perform the preset function based on the second touch input data; wherein the application processor (221) determines whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data; wherein the application processor (221) determines, in response to the application processor (221) confirming that the second touch input data indicates a stable state, a first method regarding the second touch input data to the touch sensor IC (353). The method may include an operation of transmitting information, an operation of acquiring third touch input data in the touch sensor IC (353), and an operation of determining whether to perform the preset function based on the first information and the third touch input data in the touch sensor IC (353).

[0098] According to various embodiments, the method may further include: an operation of obtaining a fourth touch input data after determining whether to perform the preset function based on the first information and the third touch input data in the touch sensor IC (353); an operation of determining whether the fourth touch input data indicates a stable state based on the third touch input data and the fourth touch input data in the touch sensor IC (353); an operation of transmitting the fourth touch input data to the application processor (221) in response to the determination that the fourth touch input data indicates an unstable state in the touch sensor IC (353); and an operation of determining whether to perform the preset function based on the fourth touch input data in the application processor (221).

[0099] According to various embodiments, the operation of determining whether the second touch input data indicates a stable state in the application processor (221) may include determining whether the second touch input data indicates a stable state based on at least one of the difference between the pixel values ​​represented by the first touch input data and the pixel values ​​represented by the second touch input data, the difference between the characteristics of the object represented by the first touch input data and the characteristics of the object represented by the second touch input data, or the difference between the result of classifying the first touch input data by a classification model and the result of classifying the second touch input data by a classification model.

[0100] According to various embodiments, the method may further include an operation of generating the classification model by machine learning in the application processor (221).

[0101] According to various embodiments, in the application processor (221), the operation of determining whether to perform a preset function based on the first touch input data includes the operation of applying a classification model generated by machine learning to the first touch input data, and in the application processor (221), the operation of determining whether to perform a preset function based on the second touch input data includes the operation of applying the classification model to the second touch input data, and the first information may include the result of applying the classification model to the second touch input data.

[0102] According to various embodiments, the operation of checking whether the fourth touch input data indicates a stable state in the touch sensor IC (353) may include checking whether the fourth touch input data indicates a stable state based on at least one of the difference between the pixel values ​​indicated by the third touch input data and the pixel values ​​indicated by the fourth touch input data, or the difference between the characteristics of the object indicated by the third touch input data and the characteristics of the object indicated by the fourth touch input data.

[0103] According to various embodiments, the method may further include the operation of entering a sleep state after transmitting the first information to the touch sensor IC (353) in the application processor (221).

[0104] According to various embodiments, the method further includes, in the application processor (221), an operation of confirming that a display off condition is satisfied, and in the application processor (221), an operation of controlling the display to an off state based on the satisfaction of the display off condition, and in the application processor (221), the display may be controlled to an off state while an operation of acquiring first touch input data from the touch sensor IC (353) is performed.

[0105] According to various embodiments, the method may further include: an operation of acquiring a fourth touch input data after determining whether to perform the preset function based on the first information and the third touch input data in the touch sensor IC (353); an operation of determining whether the fourth touch input data indicates a stable state based on the third touch input data and the fourth touch input data in the touch sensor IC (353); an operation of transmitting a wake-up signal and the fourth touch input data to the application processor (221) to cause the application processor (221) to exit a sleep state in response to the determination that the fourth touch input data indicates an unstable state; an operation of entering a wake-up state based on the wake-up signal in the application processor (221); and an operation of determining whether a display-on condition is satisfied based on the fourth touch input data in the application processor (221).

[0106] According to various embodiments, the operation of determining whether a display-on condition is satisfied based on the fourth touch input data in the application processor (221) includes applying a classification model generated by machine learning to the fourth touch input data, and the method may further include: an operation of transmitting the result of applying the classification model to the fourth touch input data to the touch sensor IC (353) based on the determination in the application processor (221) that the display-on condition is not satisfied; an operation of entering the sleep state in the application processor (221); and an operation of controlling the display to the ON state based on the determination in the application processor (221) that the display-on condition is satisfied.

[0107] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0108] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0109] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0110] Various embodiments of the present document may be implemented as software (e.g., program (340)) comprising one or more instructions stored in a storage medium (e.g., internal memory (336) or external memory (338)) readable by a machine (e.g., electronic device (301)). For example, a processor (e.g., processor (320)) of the machine (e.g., electronic device (301)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0111] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0112] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

Claim 1 An electronic device comprises a touch sensor IC configured to acquire touch input data through an application processor and a touch sensor, wherein the application processor acquires first touch input data from the touch sensor IC and determines whether to perform a preset function based on the first touch input data, acquires second touch input data from the touch sensor IC and determines whether to perform the preset function based on the second touch input data, determines whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data, determines whether the second touch input data indicates a stable state based on at least one of the difference between pixel values ​​represented by the first touch input data and pixel values ​​represented by the second touch input data, the difference between the characteristics of an object represented by the first touch input data and the characteristics of an object represented by the second touch input data, or the difference between the result of classifying the first touch input data by a classification model and the result of classifying the second touch input data by a classification model, and transmits first information regarding the second touch input data to the touch sensor IC in response to the determination that the second touch input data indicates a stable state. An electronic device configured such that the touch sensor IC acquires third touch input data and determines whether to perform the preset function based on the first information and the third touch input data. Claim 2 An electronic device according to claim 1, wherein the touch sensor IC is configured to determine whether to perform the preset function based on the first information and the third touch input data, acquire the fourth touch input data, determine whether the fourth touch input data indicates a stable state based on the third touch input data and the fourth touch input data, and transmit the fourth touch input data to the application processor in response to the determination that the fourth touch input data indicates an unstable state, and the application processor is configured to determine whether to perform the preset function based on the fourth touch input data. Claim 3 delete Claim 4 An electronic device according to claim 1, wherein the application processor is configured to determine whether to perform the preset function by applying a classification model generated by machine learning to the first touch input data and to determine whether to perform the preset function by applying the classification model to the second touch input data, and the first information includes the result of applying the classification model to the second touch input data. Claim 5 An electronic device according to paragraph 2, wherein the touch sensor IC is configured to determine whether the fourth touch input data indicates a stable state based on at least one of the difference between the pixel values ​​represented by the third touch input data and the pixel values ​​represented by the fourth touch input data, or the difference between the characteristics of the object represented by the third touch input data and the characteristics of the object represented by the fourth touch input data. Claim 6 An electronic device according to claim 1, wherein the application processor is configured to enter a sleep state after transmitting the first information to the touch sensor IC. Claim 7 In claim 6, the electronic device further comprises a display, and the application processor is configured to check that a display off condition is satisfied, control the display to an off state based on the satisfaction of the display off condition, and acquire the first touch input data from the touch sensor IC while the display is controlled to an off state. Claim 8 An electronic device according to claim 6, wherein the touch sensor IC determines whether to perform the preset function based on the first information and the third touch input data, acquires the fourth touch input data, determines whether the fourth touch input data indicates a stable state based on the third touch input data and the fourth touch input data, and in response to the determination that the fourth touch input data indicates an unstable state, transmits a wake-up signal and the fourth touch input data to the application processor to cause the application processor to exit a sleep state, and the application processor is configured to enter a wake-up state based on the wake-up signal and determine whether a display-on condition is satisfied based on the fourth touch input data. Claim 9 An electronic device configured such that, in claim 8, the application processor applies a classification model generated by machine learning to the fourth touch input data to determine whether the display on condition is satisfied, and based on the determination that the display on condition is not satisfied: transmits the result of applying the classification model to the fourth touch input data to the touch sensor IC; enters the sleep state, and, based on the determination that the display on condition is satisfied, controls the display to be in the ON state. Claim 10 A method performed in an electronic device comprising an application processor and a touch sensor IC, wherein the application processor comprises: an operation of acquiring first touch input data from the touch sensor IC; the application processor comprises: an operation of determining whether to perform a preset function based on the first touch input data; the application processor comprises: an operation of acquiring second touch input data from the touch sensor IC; the application processor comprises: an operation of determining whether to perform the preset function based on the second touch input data; the application processor comprises: an operation of determining whether the second touch input data indicates a stable state based on the first touch input data and the second touch input data; the application processor comprises: an operation of transmitting first information regarding the second touch input data to the touch sensor IC in response to the determination that the second touch input data indicates a stable state; the touch sensor IC comprises: an operation of acquiring third touch input data; and the touch sensor IC comprises: an operation of determining whether to perform the preset function based on the first information and the third touch input data. The operation of determining whether the second touch input data indicates a stable state in the application processor comprises pixel values ​​represented by the first touch input data and the second touch input A method comprising determining whether the second touch input data indicates a stable state based on at least one of the difference between pixel values ​​represented by the data, the difference between the characteristics of an object represented by the first touch input data and the characteristics of an object represented by the second touch input data, or the difference between the result of classifying the first touch input data by a classification model and the result of classifying the second touch input data by a classification model. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images