Electronic device identifying force touch and method of operating the same

Abstract

An electronic device according to various embodiments includes a memory storing a plurality of deep learning models for determining a force touch, a touch screen, and a processor. The processor is configured to check a user's touch input through the touch screen, receive touch pixel data for a plurality of frames having a time difference based on the touch input, and check whether the touch input is a force touch based on the touch pixel data. The processor is configured to check whether the touch input has been re-entered at least a specified first time within a specified time, and if it is confirmed that the touch input has been re-entered at least the specified first time within the specified time, to check whether the touch input is a force touch using a first judgment model among the plurality of deep learning models, and if it is not confirmed that the touch input has been re-entered at least the specified first time within the specified time, to check whether the touch input is a force touch using a judgment model among the plurality of deep learning models that has a lower computational amount than the first judgment model.

Description

Technology Field

[0001] Various embodiments relate to an electronic device for detecting a force touch and a method of operating the same. Background Technology

[0002] Artificial Intelligence (AI) systems are computer systems that achieve human-level intelligence. Unlike existing rule-based smart systems, they are systems that learn, make judgments, and become smarter on their own. As AI systems improve in recognition rate and become capable of understanding user preferences more accurately with continued use, existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

[0003] Artificial intelligence technology can be composed of machine learning (deep learning) and elemental technologies utilizing machine learning.

[0004] Machine learning is an algorithmic technology that classifies and learns the features of input data on its own, and elemental technology is a technology that mimics the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms such as deep learning, and can be composed of technology fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0005] The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding refers to the technology of recognizing, applying, and processing human language and text, including natural language processing, machine translation, dialogue systems, question answering, and speech recognition / synthesis. Visual understanding refers to the technology of perceiving and processing objects like human vision, including object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image enhancement. Inference and prediction refers to the technology of logically reasoning and predicting by judging information, including knowledge / probability-based inference, optimization prediction, preference-based planning, and recommendation. Knowledge representation refers to the technology of automatically processing human experiential information into knowledge data, including knowledge construction (data generation / classification) and knowledge management (data utilization). Motion control refers to the technology of controlling autonomous driving of vehicles and the movement of robots, including motion control (navigation, collision, driving) and manipulation control (behavior control).

[0006] Driven by advancements in electronic technology, various types of electronic devices are being developed and distributed. In particular, the adoption of portable electronic devices with diverse functions, such as smartphones and tablet PCs, has been expanding recently. To support the various functions of these portable electronic devices, touch input intensity (pressure) is being utilized as a new input method, in addition to conventional touch input. The problem to be solved

[0007] Conventionally, in order to determine whether a touch input is a long touch or a force touch, an electronic device used a force touch sensor (or pressure sensor) provided at the bottom of the touch screen to check the pressure of the touch input applied to the touch screen. However, since conventional electronic devices had to separately provide a force touch sensor (or pressure sensor), they could be disadvantageous in terms of cost and product miniaturization.

[0008] Recently, technology is being introduced that utilizes deep learning to determine whether a touch input is a long touch or a force touch, without the need for a force touch sensor (or pressure sensor). Deep learning models for force touch determination can be designed with high computational speeds for rapid response. However, deep learning models with high computational speeds may result in lower accuracy in force touch determination.

[0009] Various embodiments may provide an electronic device and a method of operation thereof, which selects a suitable deep learning model among a plurality of deep learning models using a history of previously determined force touches and determines whether a touch input is a long touch or a force touch through the selected deep learning model. means of solving the problem

[0010] An electronic device according to various embodiments includes a memory storing a plurality of deep learning models for determining a force touch, a touch screen, and a processor. The processor is configured to check a user's touch input through the touch screen, receive touch pixel data for a plurality of frames having a time difference based on the touch input, and check whether the touch input is a force touch based on the touch pixel data. The processor is configured to check whether the touch input has been re-entered at least a specified first time within a specified time, and if it is confirmed that the touch input has been re-entered at least the specified first time within the specified time, to check whether the touch input is a force touch using a first judgment model among the plurality of deep learning models, and if it is not confirmed that the touch input has been re-entered at least the specified first time within the specified time, to check whether the touch input is a force touch using a judgment model among the plurality of deep learning models that has a lower computational amount than the first judgment model.

[0011] A method of operation of an electronic device according to various embodiments includes, through a touch screen included in the electronic device, an operation of confirming a user's touch input, an operation of receiving touch pixel data for a plurality of frames having a time difference based on the touch input, and an operation of confirming whether the touch input is a force touch based on the touch pixel data, wherein the operation of confirming whether the touch input is a force touch includes an operation of confirming whether the touch input has been re-entered at least a specified first time within a specified time, an operation of confirming whether the touch input is a force touch using a first judgment model among a plurality of deep learning models stored in a memory included in the electronic device if it is confirmed that the touch input has been re-entered at least the specified first time within the specified time, and an operation of confirming whether the touch input is a force touch using a judgment model having a lower computational amount than the first judgment model among the plurality of deep learning models if it is not confirmed that the touch input has been re-entered at least the specified first time within the specified time.

[0012] A non-transient recording medium according to various embodiments may store a program comprising: an operation to confirm a user's touch input through a touch screen included in an electronic device; an operation to receive touch pixel data for a plurality of frames having a time difference based on the touch input; and an operation to confirm whether the touch input is a force touch based on the touch pixel data. The operation to confirm whether the touch input is a force touch includes: an operation to confirm whether the touch input has been re-entered at least a specified first time within a specified time; an operation to confirm whether the touch input is a force touch using a first judgment model among a plurality of deep learning models stored in a memory included in the electronic device if it is confirmed that the touch input has been re-entered at least the specified first time within the specified time; and an operation to confirm whether the touch input is a force touch using a judgment model having a lower computational amount than the first judgment model among the plurality of deep learning models if it is not confirmed that the touch input has been re-entered at least the specified first time within the specified time. Effects of the invention

[0013] An electronic device according to various embodiments can improve the accuracy and response speed of force touch judgment by selecting a suitable deep learning model among a plurality of deep learning models using a history of previously determined force touch. Brief explanation of the drawing

[0014] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIGS. 2a and 2b are electronic devices according to various embodiments and block diagrams thereof. FIG. 3 is a diagram illustrating the operation of an electronic device according to various embodiments acquiring touch pixel data based on touch input. FIG. 4 is a block diagram of a deep learning model for verifying a force touch according to various embodiments. FIG. 5 is a diagram illustrating the operation of an electronic device according to various embodiments confirming a force touch through a deep learning model. FIG. 6 is a diagram illustrating the operation of an electronic device according to various embodiments confirming a force touch through a deep learning model. FIG. 7 is a flowchart for explaining the operation of an electronic device according to various embodiments determining force touch through a deep learning model. FIG. 8 is a diagram showing information about multiple deep learning models according to various embodiments. FIG. 9 is a diagram illustrating the operation of an electronic device according to various embodiments determining a judgment model based on a plurality of regions included in a touch screen. FIG. 10 is a flowchart for explaining the operation of an electronic device according to various embodiments determining a judgment model based on a plurality of regions included in a touch screen. Specific details for implementing the invention

[0015] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0016] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0017] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0018] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0019] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0020] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0021] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0022] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0023] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0024] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0025] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0026] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0027] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0028] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0029] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0030] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0031] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0032] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0033] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0034] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0035] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0036] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0037] FIGS. 2a and 2b are electronic devices according to various embodiments and block diagrams thereof.

[0038] Referring to FIG. 2a, the electronic device (201) can detect a user's touch input. For example, the electronic device (201) can be implemented in the same or similar manner as the electronic device (101) of FIG. 1.

[0039] According to various embodiments, the electronic device (201) can determine whether the touch input is a long touch (or long touch input) or a force touch (or force touch input). If the electronic device (201) determines that the touch input is a force touch, it can perform a function corresponding to the force touch. For example, if the electronic device (201) determines that the touch input to an icon is a force touch, it can display a first pop-up window for the application represented by the icon. Alternatively, if the electronic device (201) determines that the touch input is a long touch, it can perform a function corresponding to the long touch. For example, if the electronic device (201) determines that the touch input to an icon is a long touch, it can display a second pop-up window for the icon. For example, the function corresponding to the force touch and the function corresponding to the long touch may be different from each other. Additionally, the content of the first pop-up window and the content of the second pop-up window may be different from each other.

[0040] According to various embodiments, the electronic device (201) can determine (or determine) whether a touch input is a force touch without using a force touch sensor or a pressure sensor. For example, the electronic device (201) can determine (or determine) whether a touch input is a force touch by using a deep learning model (or a force touch determination model using a deep learning model).

[0041] According to various embodiments, the electronic device (201) can select a deep learning model (hereinafter, judgment model) suitable for the situation among a plurality of deep learning models requiring different amounts of computation, and can check whether a touch input is a force touch using the selected judgment model. For example, the electronic device (201) can select a judgment model considering accuracy and responsiveness, and can check whether a touch input is a force touch using the selected judgment model.

[0042] Referring to FIG. 2b, the electronic device (201) may include a processor (220), memory (230), and a touch screen (260).

[0043] According to various embodiments, the processor (220) (e.g., the processor (120) of FIG. 1) can control the overall operation of the electronic device (201).

[0044] According to various embodiments, the processor (220) can detect a user's touch input through a touch screen (260) (e.g., the display module (160) of FIG. 1). For example, the touch screen (260) may include a touch IC (integrated circuit).

[0045] According to various embodiments, the processor (220) may receive (or acquire) touch pixel data for a plurality of frames having a time difference based on touch input through a touch screen (260) (or a touch IC included in the touch screen). For example, touch pixel data may refer to data for sensing values ​​that change over time due to touch input.

[0046] According to various embodiments, the processor (220) may perform a force touch determination operation based on touch pixel data. For example, the force touch determination operation may mean an operation to determine whether a touch input is a long touch or a force touch. For example, the processor (220) may determine whether a touch input is a force touch by inputting the touch pixel data as an input value to a deep learning model (e.g., a determination model) for determining a force touch.

[0047] According to various embodiments, the memory (230) (e.g., the memory (130) of FIG. 1) may store information about multiple deep learning models that require different amounts of computation. The multiple deep learning models may refer to deep learning models capable of distinguishing between a long touch and a force touch based on touch pixel data sensed by a touch input. For example, the multiple deep learning models may learn a deep learning model capable of distinguishing between a long touch and a force touch based on touch pixel data sensed by a touch input to determine whether a touch input confirmed through a touch screen is a force touch.

[0048] For example, a deep learning model that requires a large amount of computation (e.g., a first judgment model) may have high judgment accuracy for force touch. However, a deep learning model that requires a large amount of computation may have slow responsiveness. On the other hand, a deep learning model that requires a small amount of computation may have low judgment accuracy for force touch. However, a deep learning model that requires a small amount of computation may have fast responsiveness. The memory (230) may store result information (e.g., judgment result, judgment accuracy, and / or number of judgment attempts) regarding the force touch judgment confirmed using the deep learning model.

[0049] According to various embodiments, the processor (220) may determine one of a plurality of deep learning models (hereinafter, a judgment model) for determining whether the touch input is a force touch when the touch input is confirmed through the touch screen (260).

[0050] According to various embodiments, the processor (220) can check whether a touch input has been re-entered at least a specified first number of times (e.g., 2 times) within a specified time (e.g., 3 seconds). For example, the processor (220) can check whether the confirmed touch input has repeatedly requested force touch determination (or force touch determination) within the specified time. For example, the specified time may be automatically set by the processor (220) or determined by the user. Additionally, the specified first number of times may also be automatically set by the processor (220) or determined by the user.

[0051] According to various embodiments, if the processor (220) confirms that the touch input has been re-entered more than a specified first number of times within a specified time, it may perform a force touch determination operation through a first determination model having a high amount of computation (or high accuracy) among a plurality of deep learning models. For example, the processor (220) may determine whether the touch input is a force touch using the first determination model. Alternatively, if the processor (220) does not confirm that the touch input has been re-entered more than a specified first number of times within a specified time, it may determine whether the touch input is a force touch using a determination model having a lower amount of computation (or lower accuracy) than the first determination model among a plurality of deep learning models (e.g., a second determination model or a third determination model). For example, if the processor (220) confirms that the touch input has repeatedly requested a force touch determination within a specified time, it may determine whether the touch input is a force touch or a long touch using a determination model having a high amount of computation (or high accuracy).

[0052] According to various embodiments, the processor (220) may divide the display area of ​​the touch screen (260) into a plurality of regions. For example, the processor (220) may identify a first region in which a touch input is input among the plurality of regions included in the touch screen (260). The processor (220) may identify the number of times and the accuracy of judgments that were previously made in the first region of the touch screen (260) to determine whether the touch input is a force touch. Based on the number of judgments and the accuracy of judgments, the processor (220) may determine a deep learning model (or judgment model) among a plurality of deep learning models to determine whether the touch input is a force touch.

[0053] According to various embodiments, the processor (220) may perform force touch determination using a second determination model if the number of times a determination was made to determine whether a force touch was previously made in a first area where a touch input was confirmed among the entire display area (or multiple areas) of the touch screen (260) is less than a specified second number (e.g., 30 times). For example, the second determination model may require less computation than the first determination model. Additionally, the second determination model may derive a determination result more quickly than the first determination model. Alternatively, the processor (220) may perform force touch determination using a third determination model if the number of times a determination was made to determine whether a force touch was previously made in a first area where a touch input was confirmed among the entire display area (or multiple areas) of the touch screen (260) is greater than or equal to a specified second number (e.g., 30 times). For example, the third determination model may require less computation than the second determination model. In addition, the third judgment model can derive a judgment result more quickly than the second judgment model.

[0054] According to various embodiments, the processor (220) may perform a force touch determination operation through the first judgment model instead of the second judgment model or the third judgment model if the judgment accuracy for a previously identified force touch in a first area where a touch input is identified is less than a threshold value (e.g., 80%).

[0055] According to various embodiments, the processor (220) may include a CPU (or first processing unit) (221) and an NPU (or second processing unit) (223). For example, the NPU (223) can derive a judgment result quickly with lower power consumption than the CPU (221). For example, the processor (220) may run the first judgment model and the second judgment model on the CPU (221) and run the third judgment model on the NPU (223). Alternatively, the processor (220) may run only the first judgment model on the CPU (221) and run the second judgment model and the third judgment model on the NPU (223). Alternatively, the processor (220) may run the first judgment model, the second judgment model, and the third judgment model all on the CPU (221) or the NPU (223).

[0056] Through this, the processor (220) can select an efficient deep learning model (or judgment model) suitable for the situation and perform force touch determination for the user's touch input.

[0057] Meanwhile, at least some of the operations of the electronic device (201) described below may be performed by the processor (220). However, for convenience of explanation, the electronic device (201) will be described as performing the operations below.

[0058] FIG. 3 is a diagram illustrating the operation of an electronic device according to various embodiments acquiring touch pixel data based on touch input.

[0059] Referring to FIG. 3, the electronic device (201) may include a touch screen (360) (e.g., the touch screen (260) of FIG. 2b). For example, the electronic device (201) may sense a user's touch input through the touch screen (360).

[0060] According to various embodiments, touch pixel data for multiple frames having a time difference can be acquired. For example, the electronic device (201) can acquire first touch pixel data (310) corresponding to a first frame at a first time based on a user's touch input. The electronic device (201) can acquire second touch pixel data (320) corresponding to a second frame at a second time after the first time. For example, the electronic device (201) can acquire touch pixel data for multiple frames having a time difference by a single touch input. The touch pixel data corresponding to multiple frames may include sensing values ​​that are identical or different from each other for a specific pixel.

[0061] FIG. 4 is a block diagram of a deep learning model for verifying a force touch according to various embodiments.

[0062] Referring to FIG. 4, according to various embodiments, the deep learning model (or judgment model) (420) may include a convolutional neural network (CNN) module (430) and a recurrent neural network (RNN) module (440). The electronic device (201) may perform a force touch judgment operation using the deep learning model (or judgment model) (420). For example, the deep learning model may determine whether the touch input is a force touch approximately 200 to 300 ms after confirming the touch input from the user's finger. For example, the deep learning model may be implemented as a fast-responding model with a size of 1 M byte. Meanwhile, FIG. 4 illustrates only one CNN module (430) and one RNN module (440), but the technical features of the present invention may not be limited thereto. For example, the deep learning model (420) may include multiple CNN modules and RNN modules depending on the required amount of computation.

[0063] According to various embodiments, the deep learning model (420) may receive pixel touch data (410) as input values. The deep learning model (420) may process the pixel touch data (410) through at least one of a CNN module (430) and an RNN module (440) and output a determination result indicating whether the touch input is a force touch.

[0064] According to various embodiments, the first judgment model, the second judgment model, and the third judgment model can each be implemented identically or similarly to the deep learning model (420), differing only in the amount of computation.

[0065] FIG. 5 is a diagram illustrating the operation of an electronic device according to various embodiments confirming a force touch through a deep learning model.

[0066] Referring to FIG. 5, according to various embodiments, in operation 510, the electronic device (201) can detect a user's touch input through a touch screen (or touch IC) (260).

[0067] According to various embodiments, in operation 520, the electronic device (201) may acquire touch pixel data for a plurality of frames having a time difference based on a touch input through a touch screen (or touch IC) (260). For example, the touch pixel data may include data having a vertical (H), a horizontal (W), and a time (N) for the touch input. For example, the touch pixel data may include data for frames having pixel values ​​sensed on the vertical axis (H) and the horizontal axis (W) of a specified size acquired during a specified time (N).

[0068] According to various embodiments, the processor (220) may acquire touch pixel data from a touch IC. In operation 530, the processor (220) may load one of a plurality of deep learning models (or AI model) stored in memory (230). In operation 540, the processor (220) may determine whether the touch input is a force touch or a long touch based on the touch pixel data through the loaded deep learning model (or AI model).

[0069] According to various embodiments, in operation 570, the processor (220) may perform a function related to a force touch or a long touch based on a judgment result.

[0070] FIG. 6 is a diagram illustrating the operation of an electronic device according to various embodiments confirming a force touch through a deep learning model.

[0071] Referring to FIG. 6, according to various embodiments, the processor (220) may execute a driver (610), a framework (620), a deep learning model (e.g., any one of a first judgment model (630), a second judgment model (640), and a third judgment model (650)), and a force touch confirmation module (670). For example, the first judgment model (630) and the second judgment model (640) may be driven by a CPU (221). The third judgment model (650) may be driven by an NPU (223).

[0072] According to various embodiments, the driver (610), framework (620), and force touch verification module (670) may be driven by the CPU (221). For example, the driver (610) and framework (620) may be the driver and framework of the CPU (221). For example, the driver (610), framework (620), and force touch verification module (670) may be program modules driven by the CPU (221). Although FIG. 6 depicts the driver (610), framework (620), and force touch verification module (670) being placed outside the CPU (221), this is for convenience of explanation only, and the technical features of the present invention may not be limited thereto.

[0073] According to various embodiments, the driver (610) may receive touch pixel data from a touch screen (or touch IC) (260) as a touch input. For example, the driver (610) may be executed by a CPU (221). The driver (610) may transmit touch pixel data to a framework (620).

[0074] According to various embodiments, the framework (620) may include a retry check module (622) and a region check module (625). For example, the framework (620) may be executed by a CPU (221). The retry check module (622) may check whether a touch input has been re-entered more than a specified number of times within a specified time. If the retry check module (622) confirms that a touch input has been re-entered more than a specified number of times within a specified time, it may transmit touch pixel data to a first judgment model (630) driven by the CPU (221).

[0075] According to various embodiments, if the retry verification module (622) does not confirm that the touch input has been re-entered more than a specified number of times within a specified time, the area verification module (625) can identify a first area where the touch input was entered in the display area of ​​the touch screen (260). The area verification module (625) can check the number of judgments and the judgment accuracy of previously checking whether the touch input in the first area is a force touch. Based on the number of judgments and the judgment accuracy, the area verification module (625) can select a judgment model to perform a force touch judgment operation among a plurality of judgment models (630, 640, and 650).

[0076] According to various embodiments, the area verification model (625) may transmit touch pixel data to a second judgment model (640) driven by a CPU (221) if the number of judgments made to determine whether a touch input was previously a force touch in a first area where the touch input was confirmed is less than a specified second number. The area verification model (625) may transmit touch pixel data to an NPU (223) (or a third judgment model (650)) if the number of judgments made to determine whether a touch input was previously a force touch in a first area where the touch input was confirmed is greater than or equal to a specified second number. Alternatively, the area verification model (625) may transmit touch pixel data to a first judgment model (630) driven by a CPU (221) if the judgment accuracy for a previously confirmed force touch in a first area where the touch input was confirmed is less than a threshold value.

[0077] According to various embodiments, the CPU (221) or NPU (223) can determine whether a touch input is a force touch or a long touch through a judgment model selected from a plurality of judgment models (630, 640, and 650). Additionally, the CPU (221) or NPU (223) can transmit the determination result to a force touch verification module (670).

[0078] According to various embodiments, the force touch verification module (670) may perform a function assigned to a force touch or a long touch based on the determination result. Additionally, the force touch verification module (670) may store (or update) information regarding the verification result in memory (or database) (230). Subsequently, when the area verification module (625) performs a force touch determination for a touch input confirmed in the first area, it may use the verification result information stored in memory (or database) (230).

[0079] FIG. 7 is a flowchart for explaining the operation of an electronic device according to various embodiments determining force touch through a deep learning model.

[0080] Referring to FIG. 7, according to various embodiments, in operation 701, the electronic device (201) can detect a user's touch input through a touch screen (260). The electronic device (201) can obtain sensed touch pixel data based on the touch input.

[0081] According to various embodiments, in operation 703, the electronic device (201) can check whether the touch input has been re-entered more than a specified number of times (e.g., 2 times) within a specified time (e.g., 3 seconds).

[0082] According to various embodiments, if it is confirmed that a touch input has been re-entered more than a specified number of times within a specified time (e.g., operation 703), in operation 705, the electronic device (201) can determine whether the touch input is a force touch through a first judgment model among a plurality of judgment models (or a plurality of deep learning models).

[0083] According to various embodiments, if it is not confirmed that a touch input has been re-entered more than a specified number of times within a specified time (No in operation 703), the electronic device (201) can identify a first area where the touch input was entered in the display area of ​​the touch screen (260). The electronic device (201) can identify the number of times and the accuracy of the judgment that the touch input was previously identified as a force touch in the first area. In operation 705, the electronic device (201) can identify whether the number of times the touch input was previously identified as a force touch in the first area where the touch input was identified is greater than a specified second number (e.g., 30 times).

[0084] According to various embodiments, if the number of times a judgment was previously made to determine whether a touch input is a force touch in a first area where a touch input is confirmed is less than a specified second number (No in operation 707), then in operation 709, the electronic device (201) can determine whether the touch input is a force touch through a second judgment model among a plurality of judgment models (or a plurality of deep learning models).

[0085] According to various embodiments, if the number of times a determination was made whether a force touch was previously made in a first area where a touch input was confirmed is greater than or equal to a specified second number (e.g., in operation 707), in operation 711, the electronic device (201) can determine whether the determination accuracy for the force touch previously confirmed in the first area where a touch input was confirmed is greater than or equal to a threshold value (e.g., 80%).

[0086] According to various embodiments, if the judgment accuracy for a previously identified force touch in a first area where a touch input is identified is greater than or equal to a threshold value (e.g., operation 711), the electronic device (201) in operation 713 can determine whether the touch input is a force touch through a third judgment model among a plurality of judgment models (or a plurality of deep learning models). Alternatively, if the judgment accuracy for a previously identified force touch in a first area where a touch input is identified is less than a threshold value (e.g., no in operation 711), the electronic device (201) in operation 705 can determine whether the touch input is a force touch through a first judgment model among a plurality of judgment models (or a plurality of deep learning models).

[0087] According to various embodiments, in operation 715, the electronic device (201) may perform a function corresponding to a touch input based on a determination result. For example, if the touch input is confirmed to be a force touch, the electronic device (201) may perform a function designated for a force touch. Alternatively, if the touch input is confirmed to be a long touch, the electronic device (201) may perform a function designated for a long touch.

[0088] According to various embodiments, in operation 717, the electronic device (201) may store (or update) information about the judgment result in memory (230). The electronic device (201) may use the information about the judgment result stored in memory (230) when subsequently performing a force touch judgment for a touch input identified in the first area.

[0089] FIG. 8 is a diagram showing information about multiple deep learning models according to various embodiments.

[0090] Referring to FIG. 8, the electronic device (201) can select any one of a plurality of deep learning models (e.g., a first judgment model, a second judgment model, and a third judgment model) for determining whether a force touch occurs.

[0091] According to various embodiments, the first judgment model (e.g., a precision discriminator) may be driven by a CPU (221). The first judgment model may have a higher computational load (e.g., 1.5 times) than the second judgment model. The first judgment model may have a higher accuracy than the second judgment model. The first judgment model may derive a judgment result using FP16 type data. For example, the first judgment model may be applied when the number of times a force touch input was previously confirmed in the area where the touch input was confirmed is greater than or equal to a specified second number (e.g., 30 times) and the judgment accuracy is less than a specified accuracy (e.g., 80%). Additionally, the first judgment model may be applied when the touch input repeatedly requests a force touch judgment within a specified time.

[0092] According to various embodiments, a second judgment model (e.g., a basic discriminator) may be driven by a CPU (221). The second judgment model may have a lower computational load than the first judgment model. Additionally, the second judgment model may have a higher computational load than the third judgment model. The second judgment model may have a faster response than the first judgment model and a higher accuracy than the third judgment model. The second judgment model may derive a judgment result using FP16 type data. For example, the second judgment model may be applied when the number of times a force touch input was previously confirmed in an area where a touch input was confirmed is less than a specified second number.

[0093] According to various embodiments, the third judgment model (e.g., a rapid discriminator) may be run on an NPU (223). However, the third judgment model may be run on a CPU (221). The third judgment model may have a lower computational load (e.g., 0.8 times) compared to the second judgment model. Also, the third judgment model may have a smaller computational load compared to the second judgment model. The third judgment model may have a faster response rate compared to the second judgment model. The third judgment model may derive a judgment result using INT8 type data. For example, the third judgment model may be applied when the number of times a force touch input was previously confirmed in the area where the touch input was confirmed is greater than or equal to a specified second number, and the judgment accuracy is greater than or equal to a specified accuracy.

[0094] Meanwhile, the computational amount, application criteria, and data types of the judgment models described in FIG. 8 are merely exemplary, and the technical features of the present invention may not be limited thereto.

[0095] FIG. 9 is a diagram illustrating the operation of an electronic device according to various embodiments determining a judgment model based on a plurality of regions included in a touch screen.

[0096] Referring to FIG. 9, according to various embodiments, the electronic device (201) may divide the display area of ​​the touch screen (260) into a plurality of areas (e.g., 910 to 940). For example, the electronic device (201) may specify the number, shape, and / or location of the plurality of areas automatically by the processor (220) or by the user. When a user's touch input is input through the touch screen (260), the electronic device (201) can identify the area where the touch input was input among the plurality of areas (e.g., 910 to 940).

[0097] According to various embodiments, the electronic device (201) can check the number of judgments and the judgment accuracy of previously checking whether a touch input is a force touch in an area where a touch input is confirmed. For example, the electronic device (201) can use information about the area-specific verification results (or judgment results) stored in the memory (230). For example, the first area (910) may have 30 judgment attempts and a judgment accuracy of 70%. The second area (920) may have 50 judgment attempts and a judgment accuracy of 50%. The third area (930) may have 10 judgment attempts and a judgment accuracy of 90%. The fourth area (940) may have 40 judgment attempts and a judgment accuracy of 90%.

[0098] Referring to FIGS. 8 and 9, according to various embodiments, if the area where the touch input is input is a first area (910), the electronic device (201) can determine whether the touch input is a force touch through a first judgment model. If the area where the touch input is input is a second area (920), the electronic device (201) can determine whether the touch input is a force touch through a first judgment model. If the area where the touch input is input is a third area (930), the electronic device (201) can determine whether the touch input is a force touch through a second judgment model. If the area where the touch input is input is a fourth area (940), the electronic device (201) can determine whether the touch input is a force touch through a third judgment model.

[0099] Through this, the electronic device (201) can select an efficient deep learning model (or judgment model) suitable for the situation and perform force touch determination for the user's touch input. Meanwhile, the numerical values ​​regarding the number, location, shape, judgment attempt, and accuracy of the plurality of regions shown in FIG. 9 are merely exemplary, and the technical concept of the present invention may not be limited thereto.

[0100] FIG. 10 is a flowchart for explaining the operation of an electronic device according to various embodiments determining a judgment model based on a plurality of regions included in a touch screen.

[0101] Referring to FIG. 10, according to various embodiments, in operation 1001, the electronic device (201) can identify a first area where a touch input is received among a plurality of areas included in the display area of ​​the touch screen (260). For example, the electronic device (201) can divide the display area of ​​the touch screen (260) into a plurality of areas. For example, the electronic device (201) can specify the number, location, and / or shape of the plurality of areas automatically by the processor (220) or by the user. For example, the plurality of areas can be set according to the user's usage pattern. For example, the plurality of areas can be set differently depending on whether the user is a right-handed user or a left-handed user.

[0102] According to various embodiments, in operation 1003, the electronic device (201) can determine the number of times a determination was made regarding whether a force touch had previously been made in a first area and the determination accuracy of the force touch. For example, the electronic device (201) can determine the number of times a force touch was determined and the determination accuracy of the force touch for the first area by using information on the area-specific verification result (or determination result) stored in memory (230). The electronic device (201) can determine a deep learning model (or judgment model) to perform a force touch determination based on the number of confirmed determinations and the determination accuracy of the force touch.

[0104] An electronic device (201) according to various embodiments includes a memory (230) for storing a plurality of deep learning models for determining a force touch, a touch screen (260), and a processor (220). The processor is configured to check a user's touch input through the touch screen, receive touch pixel data for a plurality of frames having a time difference based on the touch input, and check whether the touch input is a force touch based on the touch pixel data. The processor is configured to check whether the touch input has been re-entered more than a specified first time within a specified time, and if it is confirmed that the touch input has been re-entered more than the specified first time within the specified time, it is configured to check whether the touch input is a force touch using a first judgment model among the plurality of deep learning models, and if it is not confirmed that the touch input has been re-entered more than the specified first time within the specified time, it is configured to check whether the touch input is a force touch using a judgment model among the plurality of deep learning models that has a lower amount of computation than the first judgment model.

[0105] The processor may be configured to identify the touch input as a long touch input or a force touch input based on the touch pixel data.

[0106] The processor may be configured to determine whether the touch input is a force touch through a second judgment model or a third judgment model among the plurality of deep learning models if it is not confirmed that the touch input has been re-entered more than the first specified number of times within the specified time.

[0107] The processor may be configured to determine whether the touch input is a force touch through the second judgment model, which has a larger amount of computation than the third judgment model, if the number of times the judgment was previously determined to determine whether the touch input is a force touch in the first area where the touch input was confirmed among the entire area of ​​the touch screen is less than the specified second number.

[0108] The processor may be configured to determine whether the touch input is a force touch through a third judgment model that has less computational power than the second judgment model if the number of times the judgment was previously determined to determine whether the touch input is a force touch in a first area where the touch input was confirmed among the entire area of ​​the touch screen is greater than or equal to a specified second number.

[0109] The processor may be configured to determine whether the touch input is the force touch through the third judgment model if the judgment accuracy for the force touch previously identified in the first region is greater than or equal to the threshold, and to determine whether the touch input is the force touch through the first judgment model if the judgment accuracy for the force touch previously identified in the first region is less than the threshold.

[0110] The above processor is an electronic device configured to drive the first judgment model and the second judgment model in a first processing unit, and to drive the third judgment model in a second processing unit.

[0111] The first judgment model and the second judgment model may be configured to process data of the first type, and the third judgment model may be configured to process data of the second type that requires less computation than the first type.

[0112] The processor may be configured to identify a first area in which the touch input is confirmed among a plurality of areas included in the display area of ​​the touch screen, and to determine the number of times the force touch was previously determined in the first area and the accuracy of the determination regarding the force touch.

[0113] The processor may be configured to check whether the touch input is the force touch, and then store information about the result of the check in the memory.

[0114] A method of operation of an electronic device (201) according to various embodiments includes, through a touch screen (260) included in the electronic device, an operation of confirming a user's touch input, an operation of receiving touch pixel data for a plurality of frames having a time difference based on the touch input, and an operation of confirming whether the touch input is a force touch based on the touch pixel data. The operation of confirming whether the touch input is a force touch may include, an operation of confirming whether the touch input has been re-entered at least once a specified first time within a specified time, an operation of confirming whether the touch input is a force touch using a first judgment model among a plurality of deep learning models stored in a memory (230) included in the electronic device if it is confirmed that the touch input has been re-entered at least once a specified first time within the specified time, and an operation of confirming whether the touch input is a force touch using a judgment model among the plurality of deep learning models that has a lower amount of computation than the first judgment model if it is not confirmed that the touch input has been re-entered at least once a specified first time within the specified time.

[0115] The operation of determining whether the touch input is the force touch may include the operation of determining the touch input as a long touch input or a force touch input based on the touch pixel data.

[0116] The operation of checking whether the touch input is a force touch may include, if the touch input is not confirmed to have been re-entered at least the first specified number of times within the specified time, the operation of checking whether the touch input is a force touch through a second judgment model or a third judgment model among the plurality of deep learning models.

[0117] The operation of checking whether the touch input is the force touch may include, if the number of times the force touch was previously checked in the first area where the touch input was checked among the entire area of ​​the touch screen is less than a specified second number, the operation of checking whether the touch input is the force touch through the second judgment model, which has a larger amount of computation than the third judgment model.

[0118] The operation of determining whether the touch input is the force touch may include, if the number of times the force touch was previously determined in the first area where the touch input was determined among the entire area of ​​the touch screen is greater than or equal to a specified second number, the operation of determining whether the touch input is the force touch through the third judgment model, which has less computational power than the second judgment model.

[0119] The operation of determining whether the touch input is the force touch may include, if the judgment accuracy for the force touch previously identified in the first area is greater than or equal to a threshold value, the operation of determining whether the touch input is the force touch through the third judgment model, and if the judgment accuracy for the force touch previously identified in the first area is less than the threshold value, the operation of determining whether the touch input is the force touch through the first judgment model.

[0120] The operation of determining whether the above touch input is the above force touch may include the operation of driving the above first judgment model and the above second judgment model in the first processing unit, and driving the above third judgment model in the second processing unit.

[0121] The first judgment model and the second judgment model may be configured to process data of the first type, and the third judgment model may be configured to process data of the second type that requires less computation than the first type.

[0122] The method of operation of the electronic device may further include the operation of checking whether the touch input is the force touch, and then storing information about the result of the check in a memory included in the electronic device.

[0123] A non-transient recording medium according to various embodiments may store a program that performs the following operations: checking a user's touch input through a touch screen (260) included in an electronic device (201); receiving touch pixel data for a plurality of frames having a time difference based on the touch input; and checking whether the touch input is a force touch based on the touch pixel data. The operation of checking whether the touch input is a force touch includes: checking whether the touch input has been re-entered at least a specified first time within a specified time; if the touch input has been re-entered at least the specified first time within the specified time, checking whether the touch input is a force touch using a first judgment model among a plurality of stored deep learning models; and if the touch input has not been re-entered at least the specified first time within the specified time, checking whether the touch input is a force touch using a judgment model among the plurality of deep learning models that has a lower computational amount than the first judgment model. Explanation of the symbols

[0125] 201: Electronic device 220: Processor 221: CPU 223: NPU 230: Memory 260: Touch screen

Claims

Claim 1 In an electronic device, a memory storing multiple deep learning models for determining force touch; a touch screen; The system includes a processor, wherein the processor confirms a user's touch input through the touch screen, receives touch pixel data for a plurality of frames having a time difference based on the touch input, and is configured to determine whether the touch input is a force touch by using a judgment result of any one of the plurality of deep learning models obtained by inputting the touch pixel data into any one of the plurality of deep learning models, wherein each of the plurality of deep learning models is trained to determine whether the touch input is a long touch or the force touch based on touch pixel data including the plurality of frames having a time difference, and the processor confirms whether the touch input has been re-entered at least a specified first time within a specified time, and if it is confirmed that the touch input has been re-entered at least the specified first time within the specified time, it uses a first judgment model among the plurality of deep learning models to determine whether the touch input is the force touch, and if it is not confirmed that the touch input has been re-entered at least the specified first time within the specified time, the first judgment model among the plurality of deep learning models An electronic device configured to determine whether the touch input is the force touch using a judgment model having a lower computational amount than the model. Claim 2 delete Claim 3 An electronic device configured such that, in the first paragraph, if the processor is not confirmed to have been re-entered more than the first specified number of times within the specified time, the processor determines whether the touch input is a force touch through a second judgment model or a third judgment model among the plurality of deep learning models. Claim 4 In paragraph 3, the processor is an electronic device configured to determine whether the touch input is the force touch through the second judgment model, which has a greater amount of computation than the third judgment model, if the number of judgments previously made to determine whether the touch input is the force touch in the first area where the touch input is confirmed among the entire area of ​​the touch screen is less than a specified second number. Claim 5 An electronic device configured such that, in the third paragraph, the processor determines whether the touch input is the force touch through the third judgment model, which has a lower computational amount than the second judgment model, if the number of judgments previously made to determine whether the touch input is the force touch in the first area where the touch input is confirmed among the entire area of ​​the touch screen is greater than or equal to a specified second number. Claim 6 An electronic device configured such that, in paragraph 5, the processor determines whether the touch input is the force touch through the third judgment model if the judgment accuracy for the force touch previously identified in the first region is greater than or equal to the threshold, and determines whether the touch input is the force touch through the first judgment model if the judgment accuracy for the force touch previously identified in the first region is less than the threshold. Claim 7 In paragraph 3, the processor is an electronic device configured to drive the first judgment model and the second judgment model in a first processing unit, and to drive the third judgment model in a second processing unit. Claim 8 An electronic device configured such that, in paragraph 3, the first judgment model and the second judgment model process data of a first type, and the third judgment model processes data of a second type that requires less computation than the first type. Claim 9 An electronic device configured such that, in claim 1, the processor identifies a first area in which the touch input is confirmed among a plurality of areas included in the display area of ​​the touch screen, and checks the number of times the force touch was previously determined in the first area and the accuracy of the determination regarding the force touch. Claim 10 An electronic device according to claim 1, wherein the processor is configured to check whether the touch input is the force touch, and then store information regarding the check result in the memory. Claim 11 A method of operating an electronic device comprises: an operation of confirming a user's touch input through a touch screen included in the electronic device; an operation of receiving touch pixel data for a plurality of frames having a time difference based on the touch input; and an operation of confirming whether the touch input is a force touch by using a judgment result of any one deep learning model obtained based on inputting the touch pixel data into any one of a plurality of deep learning models stored in the electronic device, wherein each of the plurality of deep learning models is trained to determine whether the touch input is a long touch or the force touch based on touch pixel data including the plurality of frames having a time difference, and the operation of confirming whether the touch input is the force touch comprises: an operation of confirming whether the touch input has been re-entered at least a specified first time within a specified time; and, if it is confirmed that the touch input has been re-entered at least the specified first time within the specified time, an operation of confirming whether the touch input is the force touch using a first judgment model among a plurality of deep learning models stored in a memory included in the electronic device. A method of operation of an electronic device comprising, if it is not confirmed that the touch input has been re-entered more than the first specified number of times within the specified time, a judgment model having a lower computational amount than the first judgment model among the plurality of deep learning models to determine whether the touch input is the force touch. Claim 12 delete Claim 13 A method of operation of an electronic device according to claim 11, wherein the operation of checking whether the touch input is a force touch includes, if the touch input is not confirmed to have been re-entered at least the first specified number of times within the specified time, checking whether the touch input is a force touch through a second judgment model or a third judgment model among the plurality of deep learning models. Claim 14 A method of operation of an electronic device according to claim 13, wherein the operation of checking whether the touch input is the force touch includes, if the number of times the force touch was previously checked in the first area where the touch input was checked among the entire area of ​​the touch screen is less than a specified second number, the operation of checking whether the touch input is the force touch through the second judgment model, which has a greater amount of computation than the third judgment model. Claim 15 A method of operation of an electronic device according to claim 13, wherein the operation of checking whether the touch input is the force touch includes, if the number of times the force touch was previously checked in the first area where the touch input was checked among the entire area of ​​the touch screen is greater than or equal to a specified second number, the operation of checking whether the touch input is the force touch through the third judgment model, which has a lower amount of computation than the second judgment model. Claim 16 A method of operation of an electronic device according to claim 15, wherein the operation of determining whether the touch input is the force touch comprises: an operation of determining whether the touch input is the force touch through the third judgment model if the judgment accuracy for the force touch previously determined in the first region is greater than or equal to a threshold value; and an operation of determining whether the touch input is the force touch through the first judgment model if the judgment accuracy for the force touch previously determined in the first region is less than the threshold value. Claim 17 In claim 13, the operation of determining whether the touch input is the force touch includes the operation of driving the first judgment model and the second judgment model in the first processing unit and driving the third judgment model in the second processing unit. Claim 18 A method of operation of an electronic device according to claim 13, wherein the first judgment model and the second judgment model are configured to process data of a first type, and the third judgment model is configured to process data of a second type that requires less computation than the first type. Claim 19 A method of operation of an electronic device according to claim 11, further comprising the operation of checking whether the touch input is the force touch, and then storing information about the check result in a memory included in the electronic device. Claim 20 In a non-transient recording medium, the method comprises: an operation of confirming a user's touch input through a touch screen included in an electronic device; an operation of receiving touch pixel data for a plurality of frames having a time difference based on the touch input; and an operation of confirming whether the touch input is a force touch by using a judgment result of any one deep learning model obtained based on inputting the touch pixel data into any one of a plurality of deep learning models stored in the electronic device, wherein each of the plurality of deep learning models is trained to determine whether the touch input is a long touch or a force touch based on touch pixel data including the plurality of frames having a time difference, and the operation of confirming whether the touch input is a force touch comprises: an operation of confirming whether the touch input has been re-entered more than a specified number of times within a specified time; and, if it is confirmed that the touch input has been re-entered more than a specified number of times within a specified time, an operation of confirming whether the touch input is a force touch using a first judgment model among the previously stored plurality of deep learning models. A recording medium storing a program that includes, if the touch input is not confirmed to have been re-entered more than the specified number of times within the specified time, an operation to determine whether the touch input is the force touch using a judgment model having a lower computational amount than the first judgment model among the plurality of deep learning models.

Images