Method for utilizing handwriting related to equation and electronic device thereof

Abstract

Various embodiments of the present invention comprise: a display (160); a memory (130) for storing instructions; and a processor (120), wherein the instructions, when executed by the processor, cause the electronic device to: receive handwriting including an equation; convert the handwriting into digital characters and display the converted digital characters; calculate and display a result value corresponding to the equation when no additional handwriting is input from a user for a designated time; receive a user input on the input equation; correct the equation on the basis of the user input; and recalculate and display a result value corresponding to the corrected equation. Various embodiments are possible.

Description

Method of utilizing handwriting related to mathematical formulas and the electronic device thereof

[0001] Various embodiments of the present disclosure disclose a method for utilizing handwriting related to mathematical formulas and an electronic device thereof.

[0002] With the development of digital technology, various types of electronic devices such as mobile communication terminals, PDAs (personal digital assistants), electronic notebooks, smartphones, tablet PCs (personal computers), or wearable devices are widely used. To support and enhance the functionality of these electronic devices, the hardware and / or software parts of the devices are continuously being improved.

[0003] For example, electronic devices provide a function that allows users to take necessary notes anytime and anywhere without a notebook or pen. For instance, users can write directly on a display (e.g., a touchscreen) using their hand or an electronic pen. The electronic device can receive the touch trajectory (or coordinates) of touching (or contacting) the display as input in the form of handwriting (or text). By inputting handwriting using an electronic pen, just as one would write on a notebook with a pen, users can conveniently create notes while experiencing an analog sensibility.

[0004] In one embodiment, a method and apparatus may be disclosed in which, when handwriting containing a mathematical formula is input, the handwriting is converted into digital characters and displayed, and when no additional handwriting is input for a specified period of time, a result value corresponding to the mathematical formula is calculated and displayed, and when a request for modification of the mathematical formula is input, the result value is modified and displayed in response to the modification of the mathematical formula.

[0005] An electronic device (101) according to one embodiment of the present disclosure comprises a display (160), a memory (130) for storing instructions; and a processor (120). When the instructions are executed by the processor, the electronic device may receive handwriting containing a mathematical expression, convert the handwriting into digital characters and display it, and if no additional handwriting is entered by the user for a specified period of time, calculate and display a result value corresponding to the mathematical expression, receive user input on the entered mathematical expression, modify the mathematical expression based on the user input, and recalculate and display a result value corresponding to the modified mathematical expression.

[0006] A method of operation of an electronic device (101) according to one embodiment of the present disclosure may include: receiving handwriting containing a mathematical formula; converting the handwriting into digital characters and displaying them on a display (160) of the electronic device; if no additional handwriting is entered by a user for a specified period of time, calculating and displaying a result value corresponding to the mathematical formula; receiving user input on the entered mathematical formula; modifying the mathematical formula based on the user input; and recalculating and displaying a result value corresponding to the modified mathematical formula.

[0007] According to one embodiment, by converting handwritten mathematical formulas, engineering symbols, and functions into digital characters and providing simple calculation results, it is possible to assist with note-taking in environments such as lectures.

[0008] According to one embodiment, a mathematical expression entered in handwriting can be converted into digital form in real time and provided, thereby enabling the creation of a neat note.

[0009] According to one embodiment, if there is no additional input for a specified period, an intermediate result value of the mathematical formula is provided, and if the mathematical formula is modified, the result value can be modified and provided based on the modified mathematical formula.

[0010] According to one embodiment, by recognizing a problem or mathematical expression and providing various solution processes according to the user's level of mathematics, a mathematical answer suitable for the user can be provided.

[0011] According to one embodiment, a problem within a document or image is recognized, a location to provide a solution or answer to the problem is determined, and by providing the solution or answer at the determined location, an intuitive result value can be provided to the user.

[0012] According to one embodiment, user convenience for solving math problems can be provided by recognizing handwriting within a problem sheet to generate an incorrect answer note, or by generating an incorrect answer note for a problem selected by the user.

[0013] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0014] FIG. 2 is a drawing illustrating an example of providing or modifying the result value of a mathematical formula entered in handwriting in an electronic device according to one embodiment.

[0015] FIG. 3 is a flowchart illustrating the operation method of an electronic device according to one embodiment.

[0016] FIG. 4a is a drawing illustrating an example of a symbol table recognizable in an electronic device according to one embodiment.

[0017] FIGS. 4b to 4d are drawings illustrating an example of entering an AI mathematical function in an electronic device according to one embodiment.

[0018] FIGS. 5a to 5d are drawings illustrating an example of modifying a mathematical formula in an electronic device according to one embodiment.

[0019] FIGS. 6a and 6b are drawings illustrating an example of modifying a mathematical formula in an electronic device according to one embodiment.

[0020] FIG. 7 is a flowchart illustrating a method for calculating a mathematical expression within a problem in an electronic device according to one embodiment.

[0021] FIG. 8 is a diagram illustrating an example of recognizing language within a problem in an electronic device according to one embodiment.

[0022] FIG. 9 is a flowchart illustrating a method for analyzing handwriting based on a user's mathematical level in an electronic device according to one embodiment.

[0023] FIGS. 10a to 10d are drawings illustrating an example of providing a process for solving a mathematical formula in an electronic device according to one embodiment.

[0024] FIG. 11 is a flowchart illustrating a method of recognizing a problem and providing a result value in an electronic device according to one embodiment.

[0025] FIGS. 12a and FIGS. 12b are drawings illustrating an example of an electronic device according to one embodiment recognizing a problem and providing a result value.

[0026] FIG. 13a is a drawing illustrating an example of an electronic device according to one embodiment that recognizes a problem and generates an incorrect answer note.

[0027] FIGS. 13b and FIGS. 13c are drawings illustrating an example of providing a layout optimized for device characteristics in an electronic device according to one embodiment.

[0028] FIG. 13d is a drawing illustrating an example of providing a menu for generating an incorrect answer note in an electronic device according to one embodiment.

[0029] FIG. 14 is a flowchart illustrating a method for generating an incorrect answer note by recognizing handwriting within a problem in an electronic device according to one embodiment.

[0030] FIGS. 15a and FIGS. 15b are drawings illustrating an example of an electronic device according to one embodiment that recognizes handwriting within a problem and generates an incorrect answer note.

[0031] FIGS. 16a to 16d are drawings illustrating an example of using an electronic device to solve a mathematical problem according to one embodiment.

[0032] FIGS. 17a to 17c are drawings illustrating an example of using an electronic device to solve a mathematical problem according to one embodiment.

[0033] FIGS. 18a to 18c are drawings illustrating an example of providing AI mathematical functions related to problem-solving in an electronic device according to one embodiment.

[0034] FIGS. 19a to 19f are drawings illustrating an example of controlling an AI mathematical function in an electronic device according to one embodiment.

[0035] FIGS. 20a to 20c are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0036] FIGS. 21a to 21e are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0037] FIGS. 22a to 22d are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0038] FIGS. 23a and FIGS. 23b are drawings illustrating an example of modifying a mathematical expression in an electronic device according to one embodiment.

[0039] FIGS. 24a to 24h are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0040] FIGS. 25a to 25d are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0041] FIG. 26 is a drawing illustrating an example of providing a result value for a mathematical formula in an electronic device according to one embodiment.

[0042] FIG. 27 is a diagram illustrating an example of modifying a result value by modifying a mathematical formula in an electronic device according to one embodiment.

[0043] FIG. 28 is a drawing illustrating an example in which a result value is provided differently in an electronic device according to one embodiment.

[0044] FIG. 29 is a diagram illustrating an example of entering an AI mathematical function in an electronic device according to one embodiment.

[0045] FIGS. 30a and FIGS. 30b are drawings illustrating an example of setting values ​​for converting handwriting into digital characters in an electronic device according to one embodiment.

[0046] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments.

[0047] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0048] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0049] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0050] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0051] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0052] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0053] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0054] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0055] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0056] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0057] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0058] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0059] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0060] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0061] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0062] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0063] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0064] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0065] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0066] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0067] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0068] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0069] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0070] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., by wire), wirelessly, or through a third component.

[0071] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0072] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0073] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0074] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0075] FIG. 2 is a drawing illustrating an example of providing or modifying the result value of a mathematical formula entered in handwriting in an electronic device according to one embodiment.

[0076] Referring to FIG. 2, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may receive a handwritten text (211) containing a mathematical expression, as indicated by the first reference numeral (210), and convert the handwritten text (211) into a digital character (213) to display it. The mathematical expression may include at least one of a number, a Latin symbol (or character) (e.g., A, B, C), a Greek symbol (e.g., Δ, σ, β), an operator (e.g., +, -, =, *, / ), a large operator (e.g., Σ, ∏), parentheses (e.g., (), [], {}), a delimiter (e.g., . , … ;), a mathematical expression (e.g., √, ∞), or a function (e.g., sin, cos, tan). The processor (120) can analyze the handwriting (211) to recognize (or identify) whether the handwriting (211) is a number, a Latin symbol, or a mathematical formula, and convert it into a digital character (213) based on the recognized result.

[0077] Referring to the second reference numeral (230), the processor (120) receives a handwritten text (231) containing a mathematical expression, and if no additional handwritten text is entered for a specified time (e.g., 3 seconds, 5 seconds), it can calculate the mathematical expression and provide a first result value (233) (or an intermediate result value). The processor (120) can display '=' and provide the calculation result even if the operator '=' is not entered by the user. In this case, the '=' symbol may be displayed at a certain distance from the last character (e.g., a number) of the mathematical expression that was entered (or converted into a digital character). Referring to the third reference numeral (250), the processor (120) can receive a handwritten text (251) that modifies the mathematical expression after providing a result value (253) for the mathematical expression. The processor (120) can modify a mathematical formula according to the input of handwriting (251-1, 251-2, 251-3) and recalculate a result value according to the modified mathematical formula (255) to provide a recalculated (or modified) second result value (257) (e.g., final result value).

[0078] According to one embodiment, the processor (120) may convert the first result value (233) or the second result value (257) into digital characters and provide them. Alternatively, the processor (120) may provide the first result value (233) or the second result value (257) as handwriting (e.g., user's handwriting (e.g., handwriting font)). When the processor (120) provides the first result value (233) or the second result value (257) as handwriting, the first result value (233) or the second result value (257) may be provided in a manner similar to the type of pen, size, or thickness of the handwriting.

[0079] An electronic device (101) according to one embodiment of the present disclosure comprises a display (160), a memory (130) for storing instructions; and a processor (120). When the instructions are executed by the processor, the electronic device may receive handwriting containing a mathematical expression, convert the handwriting into digital characters and display it, and if no additional handwriting is entered by the user for a specified period of time, calculate and display a result value corresponding to the mathematical expression, receive user input on the entered mathematical expression, modify the mathematical expression based on the user input, and recalculate and display a result value corresponding to the modified mathematical expression.

[0080] When the above instructions are executed by the processor, the electronic device may display the operator along with the result value corresponding to the mathematical expression if no additional handwriting is entered by the user for a specified period and no result request operator (e.g., =) is entered by the user. The operator may be displayed at a certain distance from the last character (e.g., a number) of the mathematical expression in which the handwriting was entered (or converted into a digital character).

[0081] When the above instructions are executed by the processor, the electronic device may receive user input selecting a variable within the mathematical expression, provide an inputtable variable object corresponding to the selected variable, and reflect the selected variable from the provided variable object into the mathematical expression.

[0082] When the above instructions are executed by the processor, the electronic device may analyze a note stored in the memory or an external server connected to the electronic device to determine a mathematical symbol or character corresponding to the handwriting.

[0083] When the above instructions are executed by the processor, the electronic device may analyze the stored note to determine the user's math level and determine a mathematical symbol or character corresponding to the handwriting based on the determined math level and the content of the analyzed note.

[0084] When the above instructions are executed by the processor, the electronic device may determine whether there is a mathematical symbol or character corresponding to the input mathematical expression among the stored notes, and if there is a mathematical symbol or character corresponding to the handwriting among the stored notes, determine the mathematical symbol or character corresponding to the handwriting based on the corresponding note.

[0085] When the above instructions are executed by the processor, the electronic device may obtain the age of the user based on a user account set on the electronic device, identify a math learning stage by analyzing a math expression included in the stored note, identify the frequency of use of a math expression by comparing the math expression in the stored note with the math expression included in the handwriting, determine grammatical errors based on the positional relationship of the math expression included in the handwriting, and determine the math level of the user based on at least one of the age, the math learning stage, the frequency of use, or the grammatical errors.

[0086] When the above instructions are executed by the processor, the electronic device may analyze a note stored in the memory or a server connected to the electronic device to determine the user's mathematical level, provide a result value corresponding to the mathematical formula based on the first mathematical level, provide a summary solution process and result value corresponding to the mathematical formula based on the second mathematical level, and provide a detailed solution process and result value corresponding to the mathematical formula based on the third mathematical level.

[0087] When the above instructions are executed by the processor, the electronic device may provide a visual object for changing the level of summary of the solution process and provide a changed level of summary of the solution process according to the user's selection regarding the visual object.

[0088] When the above instructions are executed by the processor, the electronic device may provide a solution process or result value on the problem when a problem corresponding to a handwriting containing the mathematical formula is obtained.

[0089] When the above instructions are executed by the processor, the electronic device may acquire a problem and a user's answer through content scanning or content retrieval, recognize the problem by deleting the user's answer from the acquired problem, and generate the recognized problem as an incorrect answer note based on the user's selection.

[0090] When the above instructions are executed by the processor, the electronic device may adjust the layout that provides the problem and the solution process corresponding to the problem based on the display size of the electronic device.

[0091] FIG. 3 is a flowchart (300) illustrating the operation method of an electronic device according to one embodiment.

[0092] Referring to FIG. 3, in operation 301, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may receive (or detect) handwriting (or writing) containing a mathematical expression. A user may input handwriting by touching the display of the electronic device (101) (e.g., display module (160) of FIG. 1) using the user's body (e.g., finger) or an electronic pen. For example, the processor (120) may execute an application for inputting handwriting (e.g., a memo application) and receive the touch trajectory (or coordinates) entered into an input area providing AI mathematical functions in the executed memo application as handwriting. Handwriting consists of strokes, and a stroke may mean a line or dot drawn once in a letter or picture (e.g., a shape). A stroke may be composed of points (e.g., feature points), and a processor (120) may obtain feature points from the stroke and analyze the stroke based on the obtained feature points.

[0093] When handwriting is input, the processor (120) can analyze the strokes of the handwriting in real time and, based on the analysis results, analyze whether the input handwriting is a character, a number, a symbol such as a mathematical symbol, or an equation. For example, a mathematical equation may include at least one of a number, a Latin symbol (e.g., A, B, C), a Greek symbol (e.g., Δ, σ, β), an operator (e.g., +, -, =, *, / ), a large operator (e.g., Σ, Π), parentheses (e.g., (), [], {}), delimiters (e.g., . , … ;), a mathematical formula (e.g., √, ∞), or a function (e.g., sin, cos, tan).

[0094] In operation 303, the processor (120) can convert and display handwriting into digital characters. The processor (120) can also convert the handwriting into a font corresponding to it. Alternatively, the processor (120) may not perform operation 303 (e.g., skip operation 303), and may keep the handwriting as is without converting it into digital characters. The digital characters may be standardized fonts (or typefaces), such as Arial, Courier New, Times New Roman, for example. Handwriting may appear somewhat unorganized and messy compared to digital characters. The processor (120) can analyze the handwriting and convert it into digital characters that are easy for the user to read, such as computer fonts. The processor (120) can analyze mathematical expressions periodically (e.g., after 1 second, 2 seconds, 3 seconds) or in real time and convert them into digital characters (e.g., text) corresponding to the mathematical expressions. The time at which the above-mentioned periodic conversion to digital characters occurs may be n seconds after the start of handwriting input, or n seconds after the time when there is no handwriting input. The processor (120) does not automatically convert the handwriting into digital characters, but can convert the handwriting into digital characters when it receives a request for conversion to digital characters from the user through a menu.

[0095] When converting to digital characters, the processor (120) can correct (or change) at least one of the lines, slant, or size of the handwriting. That is, if the horizontal lines of the input handwriting do not match, the processor (120) can correct them to be horizontal, correct the slant of the handwriting, or if the sizes of the handwriting differ, correct them to be uniform (or similar) and reflect them in the digital characters. Since converting handwriting to digital characters is a prior art, a detailed explanation may be omitted.

[0096] According to one embodiment, when the processor (120) converts a mathematical expression into digital characters, if there are unclear characters, numbers, symbols (or signs), or operators, the processor (120) may convert the mathematical expression into digital characters based on the user's mathematical level. For example, if information of a user using the electronic device (101) (e.g., name, age, grade, major, address) is stored in the electronic device (101), the processor (120) may determine the user's mathematical level (e.g., elementary school student) by considering the user's information (e.g., 10 years old) when it is unclear whether what is entered in handwriting is a character (e.g., a), a mathematical symbol (e.g., σ), or some other mathematical symbol, and may convert the handwriting into digital characters based on the determined mathematical level.

[0097] In operation 305, the processor (120) can determine whether additional handwriting is entered. For example, the processor (120) can determine whether additional handwriting is entered within a specified time (e.g., 2 seconds, 3 seconds, 5 seconds). If additional handwriting is entered, the processor (120) can return to operation 301, and if additional handwriting is not entered, it can perform operation 307. If it returns to operation 301, the processor (120) can receive the handwriting and perform an operation to convert the received handwriting into digital characters and display them.

[0098] If no additional handwriting is entered, in operation 307, the processor (120) may display a result value (or correct answer) corresponding to the mathematical expression. The processor (120) may calculate the mathematical expression entered as handwriting in operation 301 and display a result value (e.g., first result value, intermediate result value). For example, the processor (120) may convert 1+2 into digital characters, and if no additional handwriting is entered, calculate the mathematical expression for 1+2 and provide =3 as the result value. According to one embodiment, the processor (120) may determine the location where the result value is provided based on the location where the digital characters or handwriting are displayed. For example, as if the user actually entered 1+2=3, the processor (120) may display the result value next to (or alongside) where 1+2 is displayed. The processor (120) may display the result value at a certain distance from the last user input of the mathematical expression (e.g., 2).

[0099] According to one embodiment, the processor (120) may display a result value with a difference in at least one of the transparency (or shading), font, or size of the digital character and result value that corresponds to the handwriting. The processor (120) may calculate the result value of the mathematical formula entered so far if no additional handwriting, such as "=", is entered even if no handwriting requesting the result value is entered, and no additional handwriting is entered within a specified time after the calculable mathematical formula has been entered. The processor (120) may calculate the result value of the mathematical formula through an AI engine included inside the electronic device (101) and provide it to the user, or transmit the mathematical formula to an intelligent server (e.g., server (108) of FIG. 1) and receive the result value of the mathematical formula from the intelligent server and provide it to the user.

[0100] According to one embodiment, the processor (120) can analyze notes stored in memory (e.g., memory (130) of FIG. 1) or a cloud server (e.g., server (108) of FIG. 1) registered as a user account of the electronic device (101). Based on the analysis results, the processor (120) can provide a result value corresponding to a mathematical expression. For example, the processor (120) may output 1.414... as a result value for a problem regarding the square root of x where x=2, or may provide √2. Alternatively, the processor (120) may provide a decimal value of 0.5 as a result value, or provide a fractional value of ½. The processor (120) may determine the method of providing the result value based on the form frequently used by the user in note-taking or the user's major.

[0101] According to one embodiment, the processor (120) may convert the result value into a digital character and provide it, or provide it as a handwriting value (e.g., similar to the user's handwriting). The processor (120) may provide the result value similar to the pen type, size, or thickness of the handwriting.

[0102] In operation 309, the processor (120) can determine whether there is a request to modify a mathematical expression. The processor (120) can receive a request to modify a mathematical expression over the digital characters displayed in operation 303. The request to modify a mathematical expression may be to add a stroke to a minus sign (-) to change it to a plus sign (+) over the digital characters, or to add parentheses. If there is a request to modify a mathematical expression, the processor (120) performs operation 311, and if there is no request to modify a mathematical expression, it returns to operation 305 to determine whether additional handwriting is entered. The processor (120) can determine that in addition to deleting (e.g., strikethrough) or modifying (e.g., changing - to +) characters in the entered mathematical expression, continuing the mathematical expression is also included in the request to modify. For example, if a user inputs up to 1 + 2 and the result (e.g., =3) is automatically displayed, adding '+3' after 1 + 2 can be determined as a request to modify the mathematical expression.

[0103] If there is a request to modify a mathematical formula, in operation 311, the processor (120) can modify and display the result value in response to the modification of the mathematical formula. If the mathematical formula is modified, the processor (120) can recalculate the result value based on the modified mathematical formula. That is, the processor (120) can modify the result value by recalculating it according to the modified mathematical formula, rather than the result value calculated in operation 307. The user can input and modify (e.g., delete, change, cancel) the mathematical formula as if performing mathematical calculations on a notebook in an analog manner, and the processor (120) can convert the mathematical formula input by the user into digital text, or provide the result value of the mathematical formula by automatically displaying or recalculating it. According to one embodiment, the processor (120) can provide the result value by converting it into digital text or by providing it as the user's handwriting.

[0104] According to one embodiment, the processor (120) can convert and provide the result value into a suitable format based on the user's level of mathematical knowledge. For example, when x=2 and there is a question about the square root of x, instead of 1.414..., it can provide √2 (e.g., √2). Or, the processor (120) can recognize that the mathematical formula is a civil and mechanical related formula and provide ½ instead of 0.5 for the value of 5 divided by 10.

[0105] FIG. 4a is a drawing illustrating an example of a symbol table recognizable in an electronic device according to one embodiment.

[0106] Referring to FIG. 4a, a recognizable symbol table (410) in an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include numbers (401), Latin symbols, Greek symbols, operators, large operators, fences, delimiters, mathematical formulas, and functions. Numbers (401) represent numbers 0 through 9 (403), and Latin symbols may include uppercase alphabets such as A, B, C, and lowercase alphabets such as a, b, and c. Greek symbols may represent characters such as delta (Δ), omega (Ω), alpha (σ), and beta (β). Operators may include addition (+) and subtraction (-), as well as square root (√) and inequality signs (<, >). Large operators may include sigma (Σ) and permutations with repetition (∏). In addition, mathematical expressions may include infinity (∞) and set symbols (e.g., ∀, ∈, ∋).

[0107] FIGS. 4b to 4d are drawings illustrating an example of entering an AI mathematical function in an electronic device according to one embodiment.

[0108] Referring to FIG. 4b, a processor of an electronic device (101) (e.g., processor (120) of FIG. 1) may display a first user interface (420) on a display (e.g., display module (160) of FIG. 1). The first user interface (420) may be an execution screen of a memo application and may include an AI math function object (421) in a menu item. When the AI ​​math function object (421) is selected, the processor (120) may display a second user interface (430) on the display module (160). The second user interface (430) may illustrate an example of receiving pen input (431) to set an area for executing the AI ​​math function. The pen input (431) may include, for example, user input through the user's body (e.g., fingers). The third user interface (440) illustrates an example in which an AI math function execution area (441) is set by the pen input (431). In the diagram, the AI ​​math function execution area is set and illustrated, but 1) a mathematical formula can be entered without setting an area (e.g., periodically monitoring whether a mathematical formula has been entered and converting it), or 2) after handwriting input, the user can select (e.g., drag) the AI ​​math function execution area to convert it.

[0109] Referring to FIG. 4c, the fourth user interface (450) may illustrate an example of receiving handwriting (451) containing a mathematical expression within an AI mathematical function execution area. For example, the processor (120) may analyze the handwriting (451) containing a mathematical expression within the AI ​​mathematical function execution area to recognize (or identify) whether the handwriting (451) is a number, a Greek symbol, or an operator. The fifth user interface (460) is a diagram illustrating an example of handwriting (451) being converted into digital characters (461). The processor (120) may analyze the handwriting (451) and convert and display it into digital characters (461) corresponding to the handwriting (451). The processor (120) may convert the handwriting (451) into digital characters after N seconds from the time it is input (if no additional handwriting input is received within N seconds). In the fifth user interface (460), when the Delete button (463) is selected, the digital character (461) can be deleted. In the fifth user interface (460), if there is an input value (e.g., digital character (461)) in the input area, the Apply button (465) is enabled, and if there is no input value, it can be disabled. The processor (120) can receive user input (467) from outside the input area (e.g., AI math function execution area (441)).

[0110] The processor (120) may select handwriting within an input area (e.g., AI math function execution area (441)) and move / copy it outside the panel. This may be done to select only the result of a mathematical expression and place it in a desired area. The result located outside the input area may be processed as a regular pen stroke. If only the result of the mathematical expression is moved, the result within the input area may be regenerated. In formula mode, both a regular stroke and a formula stroke may be selected. If an input value that started within the input area moves outside the panel (e.g., drawn continuously with a pen), it may be visible in the area outside the input area, and upon completion of the input, the size of the input area may be expanded to include the area containing the stroke. If the type of pen is a highlighter, the highlighter may not affect the calculation because it is for emphasis purposes. That is, the highlighter may not be considered as a formula input and may be dependent on the content within the input area.

[0111] Referring to FIG. 4d, the processor (120) may display a sixth user interface (470) on the display module (160) that includes a scan function (471) or a retrieval function (473) for acquiring a note (or handwriting) containing a mathematical formula. For example, when the scan function (471) is selected, the processor (120) may run a camera application to acquire an image containing a mathematical formula through a camera (e.g., the camera module (180) of FIG. 1). Alternatively, when the retrieval function (473) is selected, the processor (120) may provide a list of images, videos, documents, or notes (or files) stored in memory (e.g., memory (130) of FIG. 1) or a cloud server (e.g., server (108) of FIG. 1) registered with the user account of the electronic device (101). The processor (120) can acquire a note (481) (or image, document) containing a mathematical formula through a scan function (471) or a retrieval function (473), and can display a seventh user interface (480) containing the acquired note (481) on a display module (160). The note (481) may contain a mathematical formula entered in handwriting. The processor (120) can analyze the mathematical formula within the note (481), convert it into a digital character (491), and calculate a result value (493) (or solution result) corresponding to the mathematical formula. The eighth user interface (490) may include the digital character (491) converted corresponding to the mathematical formula and the result value (493) corresponding to the mathematical formula.

[0112] FIGS. 5a to 5d are drawings illustrating an example of modifying a mathematical formula in an electronic device according to one embodiment.

[0113] Referring to FIG. 5a, the first user interface (510) is a diagram illustrating an example of receiving handwritten text containing a mathematical expression from a user within the first AI mathematical function area (511). The amount of handwritten text within the first AI mathematical function area (511) may increase, or the first AI mathematical function area (511) may be expanded (or increased) according to the user's selection. The second user interface (520) is a diagram illustrating an example of receiving handwritten text containing a mathematical expression from a user within the second AI mathematical function area (521). The second AI mathematical function area (521) represents an example in which the size is expanded (or increased) compared to the first AI mathematical function area (511).

[0114] Referring to FIG. 5b, a third user interface (530) is illustrated as an example of receiving user input (531) for modifying a mathematical expression. A fourth user interface (540) is illustrated as an example of modifying a mathematical expression based on user input for modifying a mathematical expression and calculating and providing a result value (541) based on the modified mathematical expression. A processor (e.g., processor (120) of FIG. 1) may calculate and provide a result value based on the input mathematical expression if no user input is detected for modifying (e.g., deleting, changing, canceling) or adding a mathematical expression for a specified period of time.

[0115] Referring to FIG. 5c, the fifth user interface (550) illustrates an example of receiving user input (551) (e.g., scribbling, drawing) for modifying a mathematical expression. The sixth user interface (560) illustrates an example of modifying a mathematical expression based on user input for modifying a mathematical expression and calculating and providing a result value (561) based on the modified mathematical expression.

[0116] Referring to FIG. 5d, the seventh user interface (570) illustrates an example of selecting a first user input (571) containing a mathematical expression and a second user input (573) not containing a mathematical expression. The processor (120) may provide separate visual effects to the input area containing the mathematical expression and may provide AI mathematical function objects (587-1, 587-2, 687-3). Referring to the eighth user interface (580), in addition to the AI ​​mathematical function object (587-3) which can activate a plurality of AI mathematical function execution areas at the bottom, the first AI mathematical function object (587-1) and the second AI mathematical function object (587-2) may be displayed respectively adjacent to the handwriting recognized as a mathematical expression in the input area. The eighth user interface (580) illustrates an example of selecting a first user input (581) containing a first mathematical expression, a second user input (583) not containing a mathematical expression, and a third user input (585) containing a second mathematical expression. The ninth user interface (590) illustrates an example of activating an AI mathematical function area as an AI mathematical function object (587) is selected in the eighth user interface (580). The first AI mathematical function area (591) may be an area containing a first mathematical expression, and the second AI mathematical function area (593) may be an area containing a second mathematical expression.

[0117] FIGS. 6a and 6b are drawings illustrating an example of modifying a mathematical formula in an electronic device according to one embodiment.

[0118] Referring to FIG. 6a, the first user interface (610) may illustrate an example of receiving a first user input (611) that modifies a mathematical expression. The first user input (611) may be selecting a result value of the mathematical expression. The second user interface (620) may illustrate an example of providing a first inputtable variable object (621) based on the first user input. According to one embodiment, the processor (120) may provide the first inputtable variable object (621) as a digital character or as the user's handwriting.

[0119] A processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may identify (or determine) the user's intention by analyzing the mathematical formula and the location (e.g., result value) where the first user input (611) is detected on the mathematical formula. Based on the analysis result, the processor (120) may determine that the user's intention is to change the result value, which is expressed as a decimal point, into a fractional value. The processor (120) may provide a fractional value corresponding to the decimal point as a first inputtable variable object (621). When the first inputtable variable object (621) is selected, the processor (120) may display a third user interface (630) in which the result value is changed from a decimal point to a fractional value (631) on a display (e.g., display module (160) of FIG. 1).

[0120] Referring to FIG. 6b, the fourth user interface (650) may illustrate an example of receiving a second user input (651) that modifies a mathematical expression. The second user input (651) may be selecting an operator (e.g., =) or a result value. The fifth user interface (660) may illustrate an example of providing a second inputtable variable object (661) based on the second user input. The processor (120) may identify (or determine) the user's intent by analyzing the mathematical expression and the location (e.g., result value) where the second user input (651) is detected on the mathematical expression. Based on the analysis result, the processor (120) may determine that the user's intent is to change the unit (e.g., cm) of the result value (e.g., m). The processor (120) may provide the result value with the unit changed to m (e.g., 0.04 m) as the second inputtable variable object (661). When the second inputtable variable object (661) is selected, the processor (120) can display the sixth user interface (670) on the display module (160), which is changed to a result value (671) in which the result value is changed in m units. According to one embodiment, the processor (120) can provide the second inputtable variable object (661) as a digital character or a user's handwriting.

[0121] FIG. 7 is a flowchart (700) illustrating a method for calculating a mathematical expression within a problem in an electronic device according to one embodiment.

[0122] Referring to FIG. 7, in operation 701, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment can analyze an image. Although labeled as an image in the drawing, the image may include any form of data obtainable through a scan function or a retrieval function. For example, the processor (120) can analyze files such as videos, notes, and documents instead of the image. The processor (120) can analyze an image displayed on a display (e.g., display module (160) of FIG. 1) to recognize (or obtain) a problem or mathematical expression within the image.

[0123] In operation 703, the processor (120) can analyze language. The processor (120) can analyze characters included in the image and identify the language of the analyzed characters. For example, the processor (120) can recognize characters in the image through optical character recognition (OCR) and determine whether the recognized characters are Korean, English, or Chinese.

[0124] In operation 705, the processor (120) can analyze the type of problem. The type of problem may include whether it is multiple choice or short answer. The processor (120) can determine whether the problem contains shapes, images, or tables. The mathematical formulas in the problem may vary, and depending on the type of mathematical formula, the AI ​​(artificial intelligence) engine included in the electronic device (101) can solve the mathematical formula on its own. The AI ​​engine included in the electronic device (101) may refer to an artificial intelligence neural network that generates new forms of data based on user input information. If the mathematical formula is somewhat difficult, the AI ​​engine cannot solve it, but the AI ​​engine can solve mathematical formulas below a certain level.

[0125] In operation 707, the processor (120) can determine whether the electronic device (101) can compute. To do this, the processor (120) can analyze the mathematical expression to determine whether the AI ​​engine can process it. Alternatively, the processor (120) can transmit the mathematical expression to the AI ​​engine and identify whether a result value is received (or obtained) from the AI ​​engine. If the electronic device (101) can compute it, the processor (120) can perform operation 711, and if the electronic device (101) cannot compute it, it can perform operation 709.

[0126] If calculation is not possible in the electronic device (101), in operation 709, the processor (120) can calculate the mathematical expression through an intelligent server (e.g., the server (108) of FIG. 1). The server (108) is a server existing outside the electronic device (101) and may provide processing results using machine learning and / or a neural network. The processor (120) can transmit the problem recognized through image analysis to the server (108) via a communication module (e.g., the communication module (190) of FIG. 1) and receive (or obtain) the calculation result (e.g., result value) or solution process from the server (108) via the communication module (190). The processor (120) can provide the calculation result or solution process to the user based on the language recognized in the problem. For example, if the language recognized in the problem is English, the processor (120) can provide the calculation result or solution process in English. Alternatively, if the language recognized in the problem is English and the language set in the electronic device (101) is Korean, the processor (120) may provide the calculation result or the solution process in Korean regardless of the language recognized in the problem. The processor (120) may display the calculation result or the solution process on the display module (160) or output it to a speaker (e.g., the sound output module (155) of FIG. 1).

[0127] In operation 711, if computation is possible in the electronic device (101), the processor (120) can calculate a mathematical expression through the AI ​​engine. The processor (120) can obtain the calculation result or the solution process from the AI ​​engine and provide it to the user. The processor (120) can display the calculation result or the solution process on the display module (160) or output it to the sound output module (155). The processor (120) can provide the calculation result or the solution process to the user based on the language recognized in the problem. For example, if the language recognized in the problem is Chinese, the processor (120) can provide the calculation result or the solution process in Chinese. Or, if the language recognized in the problem is Chinese and the language set in the electronic device (101) is English, the processor (120) can provide the calculation result or the solution process in English regardless of the language recognized in the problem.

[0128] FIG. 8 is a diagram illustrating an example of recognizing language within a problem in an electronic device according to one embodiment.

[0129] Referring to FIG. 8, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment can analyze all forms of data obtainable through a scan function or a retrieval function, such as files (or content) such as images, videos, notes, and documents. The processor (120) can recognize (or obtain) a problem or mathematical expression by analyzing data displayed on a display (e.g., display module (160) of FIG. 1). The processor (120) can recognize whether the language within the recognized problem is Korean (810), English (820), Japanese (850), or Chinese (870).

[0130] According to one embodiment, the processor (120) may translate the problem based on the language of the problem or the language set on the electronic device (101) and provide it to the user. For example, if the language of the problem is Chinese and the language set on the electronic device (101) is Korean, the processor (120) may translate the problem from Chinese into Korean and provide it to the user. Alternatively, the processor (120) may provide the correct answer or solution process for the problem based on the language of the problem or the language set on the electronic device (101). For example, if the language of the problem is English and the language set on the electronic device (101) is Korean, the processor (120) may provide the correct answer or solution process for the problem in English. Alternatively, if the language of the problem is English and the language set on the electronic device (101) is Korean, the processor (120) may provide the correct answer or solution process for the problem in Korean.

[0131] FIG. 9 is a flowchart (900) illustrating a method for analyzing handwriting based on the user's mathematical level in an electronic device according to one embodiment.

[0132] Referring to FIG. 9, in operation 901, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may receive (or detect) handwriting containing a mathematical expression. A user may input handwriting by touching the display of the electronic device (101) (e.g., display module (160) of FIG. 1) using the user's body (e.g., finger) or an electronic pen. For example, the processor (120) may execute an application for inputting handwriting containing a mathematical expression and receive a touch trajectory (or coordinates) entered into an input area that provides AI mathematical functions in the executed application as handwriting. Handwriting consists of strokes, and a stroke may mean a line or dot drawn once in writing or a picture (e.g., a shape). When handwriting is input, the processor (120) can analyze the strokes of the handwriting in real time and, based on the analysis results, analyze whether the input handwriting is a character, a number, a symbol such as a mathematical symbol, or an equation. For example, a mathematical equation may include at least one of a number, a Latin symbol (e.g., A, B, C), a Greek symbol (e.g., Δ, σ, β), an operator (e.g., +, -, =), a large operator (e.g., Σ, Π), parentheses (e.g., (), [], {}), delimiters (e.g., . , … ;), a mathematical formula (e.g., √, ∞), or a function (e.g., sin, cos, tan).

[0133] In operation 903, the processor (120) can analyze the stored notes. Operation 903 of analyzing the stored notes may be performed before operation 901. That is, the processor (120) may perform operation 901 of receiving handwriting after analyzing the stored notes (e.g., operation 903). The invention is not limited by the order of operations. The processor (120) can analyze notes stored in memory (e.g., memory (130) of FIG. 1) or a cloud server (e.g., server (108) of FIG. 1) registered as a user account of the electronic device (101).

[0134] In operation 905, the processor (120) can determine whether there are mathematical symbols (or signs) or characters corresponding to handwriting in the stored note. Since handwriting is input based on touch trajectories, the processor (120) may not recognize what kind of characters the handwriting is when receiving handwriting input from a user who is not good at writing (e.g., illegible handwriting). Taking this into consideration, the processor (120) can determine the mathematical symbols or characters corresponding to the handwriting based on notes that the user has previously entered. The processor (120) can perform operation 907 if there are mathematical symbols or characters corresponding to the handwriting in the stored note, and perform operation 911 if there are no mathematical symbols or characters corresponding to the handwriting in the stored note. Even if there are mathematical symbols or characters corresponding to the handwriting, the processor (120) can additionally determine the user's learning comprehension level (e.g., age, math learning stage, etc.).

[0135] In operation 907, if there is a mathematical symbol or character corresponding to the handwriting in the stored note, the processor (120) can determine the mathematical symbol or character corresponding to the handwriting based on the note. For example, if it is unclear whether the handwriting is a or σ, the processor (120) can determine the handwriting as either a or σ based on the stored note. Or, if it is unclear whether the handwriting is 0 or σ, the processor (120) can determine the handwriting as either σ or 0 based on the stored note.

[0136] In operation 909, the processor (120) can convert and display handwriting into digital characters. The digital characters may be standardized fonts (or typefaces), such as Arial, Courier New, Times New Roman, for example. The processor (120) can analyze the handwriting and convert it into digital characters that are easy for the user to read, such as computer typefaces. The processor (120) can analyze mathematical expressions periodically (e.g., after 1 second, 2 seconds, 3 seconds) or in real time and convert them into digital characters (e.g., text) corresponding to the mathematical expressions. When converting to digital characters, the processor (120) can correct (or change) at least one of the lines, slant, or size of the handwriting. The processor (120) can perform operations 901 through 909 when performing operations 301 and 303 of FIG. 3.

[0137] In operation 911, if there are no mathematical symbols or characters corresponding to handwriting in the above-mentioned stored notes, the processor (120) can determine the user's mathematical level and / or field of detail.

[0138] According to one embodiment, the electronic device (101) may store user information (e.g., name, age, grade, major, address). Alternatively, a user account of the electronic device (101) may be registered, and the processor (120) may obtain user information based on the user account. The processor (120) may obtain user information based on information stored in memory (130) or a server (108). The processor (120) may obtain user information as one of the methods for determining the user's level of mathematics and / or specific field.

[0139] According to one embodiment, the processor (120) can identify a math learning stage. The processor (120) can estimate the math learning stage by analyzing notes stored in memory (130) or server (108). The processor (120) can receive the math learning stage from the user or set the math learning stage based on the user's information (e.g., age or grade). The processor (120) can output different answer results depending on the user's major field (e.g., civil engineering, architecture). Alternatively, the processor (120) can identify the user's math learning stage based on information stored in memory (130) or server (108) (e.g., worksheets, test papers). The processor (120) can identify the math learning stage as one of the methods for determining the user's math level and / or specific field.

[0140] According to one embodiment, the processor (120) can identify the frequency of use of mathematical formulas. The processor (120) can identify mathematical formulas contained within a stored note and determine whether there is a frequency of use of a specific mathematical formula or whether there are characters (e.g., Latin symbols, Greek symbols) that are repeatedly used in a specific mathematical formula. The processor (120) can count the frequency of use of the repeatedly used characters and store them in memory (130). The processor (120) can identify the frequency of use of mathematical formulas as one of the methods for determining the user's level of mathematics and / or field of study.

[0141] According to one embodiment, the processor (120) can determine grammatical errors in a mathematical expression. There may be items that must be entered in the mathematical expression. For example, in sigma, symbols (e.g., x, y) and variables must be entered, and it can determine whether these items are omitted. The processor (120) can determine grammatical errors in the mathematical expression as one of the methods for determining the user's level of mathematics and / or field of study.

[0142] According to one embodiment, the processor (120) can determine the user's math level. For example, the processor (120) can determine the user's math level by considering at least one of the user's age, grade level, major, math learning stage, frequency of use of math formulas, or grammatical errors in math formulas. For example, if the user is 8 years old, the processor (120) can determine the math level as that of a lower elementary grade, and if the user is 16 years old, the processor can determine the math level as that of a high school. Alternatively, even if the math level is that of a lower elementary grade, the processor (120) can further consider the math learning stage to divide the math level more specifically. For example, the processor (120) can divide the math level of a lower grade into upper / lower or upper / middle / lower in detail.

[0143] The processor (120) may sequentially perform actions to identify the user's age, grade, major, level of mathematical learning, frequency of use of mathematical expressions, or grammatical errors of mathematical expressions as one of the methods for determining the user's mathematical level, or may omit some actions.

[0144] In operation 913, the processor (120) may determine a mathematical symbol or character corresponding to the handwriting based on the determined user's mathematical level and / or specific field. For example, if it is unclear whether the handwriting is 0 or σ, the processor (120) may determine the handwriting as either σ or 0 based on the user's mathematical level. The processor (120) may determine the handwriting as 0 if the user's mathematical level is in the lower grades of elementary school. Or, the processor (120) may determine the handwriting as σ if the user's mathematical level is in the lower grades of middle school. The processor (120) may determine the handwriting as σ if the user's major is civil engineering. Or, the processor (120) may determine a mathematical symbol of a form frequently used in the user's occupational group (e.g., architecture).

[0145] FIGS. 10a to 10d are drawings illustrating an example of providing a process for solving a mathematical formula in an electronic device according to one embodiment.

[0146] Referring to FIGS. 10a and 10b, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may recognize a problem (1000) containing a mathematical expression and solve the problem (1000) to provide a correct answer. When providing a correct answer, the processor (120) may provide various solution processes considering the user's mathematical level. For example, if the user's mathematical level is high, the processor (120) may provide only the correct answer (1010), and if the user's mathematical level is low, it may provide a first correct answer solution process (1020), a second correct answer solution process (1030), a third correct answer solution process (1040), or a fourth correct answer solution process (1050) depending on the user's mathematical level. The first answer solution process (1020), the second answer solution process (1030), and the third answer solution process (1040) are summaries of the fourth answer solution process (1050), and may indicate that the summaries increase as one moves from the third answer solution process (1040) to the first answer solution process (1020). For example, the processor (120) may provide only the correct answer (1010) when the user's math level is Grade 1, provide the first answer solution process (1020) when the user's math level is Grade 2, provide the second answer solution process (1030) when the user's math level is Grade 3, provide the third answer solution process (1040) when the user's math level is Grade 4, and provide the fourth answer solution process (1050) when the user's math level is Grade 5.

[0147] Referring to FIG. 10c, the processor (120) may provide a visual object (1005) for changing the level of summary of the solution process in the first solution process (1020). The visual object (1005) is intended to change the level of summary of the solution process by moving the circular shape of the slider bar to the left or right, such as a slider bar. For example, the processor (120) may provide a simple solution process (e.g., correct answer (1010)) when the circular shape of the slider bar is moved to the left based on user input in the first solution process (1020), and provide a detailed solution process (e.g., second correct answer solution process (1030), third correct answer solution process (1040), or fourth correct answer solution process (1050)) when the circular shape of the slider bar is moved to the right. Alternatively, the processor (120) may detect user input in the first solution process (1020) that lowers the bottom of the area of ​​the first solution process (1020). The processor (120) may provide a third correct answer solution process (1040) based on the user input that lowers the bottom of the area of ​​the first solution process (1020).

[0148] Referring to FIG. 10d, the processor (120) can detect a user input (1031) hovering over a part of the content included in the second answer solution process (1030). In this case, the processor (120) can indicate that the mathematical expression (1061) for which the user input (1031) was detected is selected. For example, as shown in the first reference numeral (1060), the processor (120) can display the mathematical expression (1061) for which the user input (1031) was detected in a different color (or bold). Alternatively, as shown in the second reference numeral (1070), the processor (120) can separately provide a detailed solution process (1071) related to the mathematical expression (1061) for which the user input (1031) was detected.

[0149] FIG. 11 is a flowchart (1100) illustrating a method of recognizing a problem and providing a result value in an electronic device according to one embodiment.

[0150] Referring to FIG. 11, in operation 1101, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment can analyze a mathematical problem. The processor (120) can recognize a mathematical problem from an image (or video, note, document) obtained through a scan function or a retrieval function, and can analyze the recognized mathematical problem.

[0151] In operation 1103, the processor (120) can calculate a mathematical problem. The processor (120) can calculate the mathematical problem through an electronic device (101) or an intelligent server (e.g., server (108) of FIG. 1). The processor (120) can determine whether the analyzed mathematical problem can be processed by an AI engine. Alternatively, the processor (120) can transmit the mathematical problem to the AI ​​engine and identify whether a result value is received (or obtained) from the AI ​​engine. If the problem can be calculated by the electronic device (101), the processor (120) can calculate the mathematical problem through the AI ​​engine. Alternatively, the processor (120) can transmit the recognized mathematical problem to the server (108) through a communication module (e.g., communication module (190) of FIG. 1) and receive (or obtain) the calculation result (e.g., result value) or solution process from the server (108) through the communication module (190).

[0152] In operation 1105, the processor (120) can determine the location for displaying the result value. The processor (120) may provide the result value above the math problem, or provide the result value at a location that does not overlap with the math problem. If the location where the result value must be entered in the math problem is fixed (e.g., a blank space in a subjective question), the processor (120) may determine that the result value is displayed at the fixed location. Alternatively, the processor (120) may provide the result value above the math problem to improve the understanding of the math problem. The result value may include the solution process, and in order to make it easier for the user to understand how the result value was derived, the processor (120) may provide the result value above the math problem.

[0153] In operation 1107, the processor (120) may display a result value or a solution process at the result value display location. The processor (120) may provide a result value or a solution process for a problem recognized in operation 1101. If the location where a result value must be entered in a math problem is fixed (e.g., a blank space in a subjective question), the processor (120) may display a result value at a fixed location based on the result value display location.

[0154] FIGS. 12a and FIGS. 12b are drawings illustrating an example of an electronic device according to one embodiment recognizing a problem and providing a result value.

[0155] Referring to FIG. 12a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment can recognize a first mathematical problem (1210). The processor (120) can recognize the first mathematical problem (1210) from an image (or video, note, document) obtained through a scan function or a retrieval function. The first mathematical problem (1210) is a multiple-choice question and may include text (1211), a graph (1213), or candidate answers (1215) (e.g., five) that describe the problem. The processor (120) can calculate the first mathematical problem (1210) by analyzing the first mathematical problem (1210). The processor (120) can calculate the first mathematical problem (1210) through an AI engine or intelligent server (e.g., the server (108) of FIG. 1) included in the electronic device (101). The processor (120) can determine the result value display location.

[0156] Referring to the first drawing symbol (1220), the processor (120) may display a result value (1221) or a solution process (1223, 1225, 1227) on a mathematical problem. The processor (120) may determine the position of the result value display location on the mathematical problem. The result value (1221) may be selecting the third correct answer candidate from the correct answer candidates. The first solution process (1223) may mean a mathematical solution process. The second solution process (1225) and the third solution process (1227) are marking lines and numbers on the graph of the problem, which can enhance the user's understanding compared to simply providing the first solution process (1223).

[0157] Referring to FIG. 12b, the processor (120) can recognize a second mathematical problem (1230). The processor (120) can recognize the second mathematical problem (1230) from an image (or video, note, document) obtained through a scan function or a retrieval function. The second mathematical problem (1230) is a subjective problem and may include text describing the problem and an area for entering the answer. The processor (120) can analyze the second mathematical problem (1230) and calculate the second mathematical problem (1230). The processor (120) can calculate the second mathematical problem (1230) through an AI engine included in the electronic device (101) or an intelligent server (e.g., the server (108) of FIG. 1). Additionally, the processor (120) can determine the result value display location.

[0158] Referring to the second reference numeral (1240), the processor (120) can display a result value at each result value display location (1241, 1243, 1245). The processor (120) can determine a blank space within a math problem at the result value display locations. The processor (120) can display '-1' as the first result value in the first blank space (or first result value display location) (1241), display '2' as the second result value in the second blank space (1243), and display '2' as the third result value in the third blank space (1245).

[0159] FIG. 13a is a drawing illustrating an example of an electronic device according to one embodiment that recognizes a problem and generates an incorrect answer note.

[0160] Referring to FIG. 13a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may display a first user interface (1310) containing a plurality of mathematical problems on a display (e.g., display module (160) of FIG. 1). The first user interface (1310) may include a plurality of mathematical problems obtained from an image (or video, note, document) obtained through a scan function or a retrieval function. The processor (120) may recognize a plurality of mathematical problems (e.g., 10 problems) from the image. The plurality of mathematical problems may include the user's handwriting. When a handwriting deletion object (1311) is selected in the first user interface (1310), the processor (120) may remove the user's handwriting present on the mathematical problems. When Add Photo (1315) is selected in the first user interface (1310), the processor (120) may provide an image (or image list) stored in memory (e.g., memory (130) of FIG. 1) or a cloud server (e.g., server (108) of FIG. 1).

[0161] When the processor (120) selects the area addition object (1313) in the first user interface (1310), it can display the second user interface (1320) on the display module (160). The second user interface (1320) may represent an example of selecting a desired math problem (e.g., ①, ②, ③, ④) from among a plurality of math problems. Since the processor (120) recognizes 10 math problems in the first user interface (1310), it can display an area containing each math problem in the second user interface (1320). The user can select an area of ​​a desired math problem or directly add an area. After the processor (120) selects a math problem, when the user selects the completion object (1321), it can create an incorrect answer note with the selected math problem.

[0162] FIGS. 13b and FIGS. 13c are drawings illustrating an example of providing a layout optimized for device characteristics in an electronic device according to one embodiment.

[0163] Referring to FIGS. 13b and 13c, the processor (120) can provide a layout optimized for the device characteristics by reflecting the device characteristics of the electronic device (101). For example, if the electronic device (101) is a device with a large screen size, such as a tablet PC (or pad), the processor (120) can provide a layout of mathematical problems such as a third user interface (1330). The third user interface (1330) may illustrate an example in which a mathematical problem (1331) is placed on the left and an empty area (1333) is provided on the right. In this case, the user can input handwriting in the empty area (1333) to the right of the mathematical problem (1331) displayed on the left, so as to feel like writing on a real notebook. Alternatively, if the electronic device (101) is a device with a smaller screen size than a tablet PC, such as a smartphone, the processor (120) may provide a layout of a mathematical problem such as a fourth user interface (1340) or a fifth user interface (1350). The fourth user interface (1340) may illustrate an example in which a mathematical problem is placed to fill the screen, with an empty area in the space below to provide a solution process (1341). The fifth user interface (1350) may illustrate an example in which a result value or a solution process for a mathematical problem is provided as a pop-up window (1351) above the mathematical problem.

[0164] FIG. 13d is a drawing illustrating an example of providing a menu for generating an incorrect answer note in an electronic device according to one embodiment.

[0165] Referring to FIG. 13d, the processor (120) may display a sixth user interface (1370) that provides AI math functions related to problem solving on the display module (160). When a sorting menu object (1371) (e.g., image, icon) is selected in the sixth user interface (1370), the processor (120) may provide a first pop-up window (1373). The first pop-up window (1373) may sort or group math problems by number, important problems, or incorrect problems. In the number order, the number may refer to the problem number. Important problems may refer to problems containing handwriting (e.g., a star shape) indicating that they are important problems. Incorrect problems may refer to problems containing handwriting (e.g., a non mark) indicating that they are incorrect problems. The electronic device (101) may provide various functions to allow the user to easily create incorrect answer notes. When the processor (120) selects the Try Solving menu object (1375) in the sixth user interface (1370), it can provide the solution process for the problem to a second pop-up window (1377). The second pop-up window (1377) may include the correct answer and detailed (or summary) solution process for the math problem.

[0166] FIG. 14 is a flowchart (1400) illustrating a method of generating an incorrect answer note by recognizing handwriting in a problem in an electronic device according to one embodiment.

[0167] Referring to FIG. 14, in operation 1401, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may acquire an image containing a question paper and an answer. The processor (120) may acquire an image (or video, note, or document) acquired through a scan function or a retrieval function. Although described as an image in the drawing, it may include notes, videos, or documents in addition to the image.

[0168] In operation 1403, the processor (120) can analyze the incorrect answer. The processor (120) can recognize a problem (or mathematical problem) from the acquired image and analyze the recognized problem. The processor (120) can calculate the correct answer to the problem based on the analysis result. Additionally, the processor (120) can recognize the user's handwriting from the acquired image and analyze the recognized handwriting. In the case of a multiple-choice question, the processor (120) can determine (or identify) whether the user has selected the correct answer by analyzing the location where the handwriting was entered among the multiple-choice answer candidates. Alternatively, in the case of a subjective question, the processor (120) can determine whether the user has entered the correct answer by analyzing the handwriting regarding the answer or the solution process.

[0169] In operation 1405, the processor (120) can classify problems corresponding to incorrect answers. The processor (120) can recognize problems and distinguish incorrect problems by analyzing handwriting detected on the recognized problems.

[0170] In operation 1407, the processor (120) can detect user input for selecting a problem. The processor (120) can include not only the classified incorrect problems but also problems selected by the user in the incorrect answer note. The processor (120) can receive a selection of problems to be included in the incorrect answer note based on user input. The user may include problems that are judged to be important or problems that were answered correctly but judged to be difficult in the incorrect answer note. Instead of receiving a problem selection directly from the user, the processor (120) can classify (or distinguish) important problems (e.g., star-shaped input in handwriting) or difficult problems (e.g., triangle-shaped input) through handwriting analysis and receive confirmation from the user whether to select the classified problems. The processor (120) can omit operation 1407 if it creates an incorrect answer note using only the classified incorrect problems.

[0171] In operation 1409, the processor (120) can generate an incorrect answer note. The incorrect answer note may include not only the incorrect problem but also the problem selected by the user. The incorrect answer note may be stored in association with a note application or in association with a separate application (e.g., an incorrect answer note application, a gallery application, a document application). When the note application is executed based on user input, the processor (120) may provide the incorrect answer note along with other notes.

[0172] FIGS. 15a and FIGS. 15b are drawings illustrating an example of an electronic device according to one embodiment that recognizes handwriting within a problem and generates an incorrect answer note.

[0173] Referring to FIG. 15a, the first user interface (1510) may illustrate an example of acquiring an image through a scan function or a load function. A processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may acquire an image (or video, note, document) through a scan function or a load function. The second user interface (1520) may illustrate an example of removing handwriting on an image. When a handwriting deletion object (1521) is selected, the processor (120) provides a handwriting removal object (1523), and can remove the handwriting as the handwriting removal object (1523) moves. In the second user interface (1520), the area above the handwriting removal object (1523) may indicate a state where the handwriting has been removed, and the area below the handwriting removal object (1523) may indicate a state where the handwriting has not been removed and remains.

[0174] Referring to FIG. 15b, when a region addition object (1525) is selected in the second user interface (1520), the processor (120) can display a third user interface (1530) on a display (e.g., the display module (160) of FIG. 1). The processor (120) can recognize mathematical problems from an image and provide a region for each mathematical problem. The third user interface (1530) is a diagram illustrating an example in which a region is designated (or set) for each mathematical problem. The user can select a problem to be included in an incorrect answer note by selecting (e.g., touching) a + object within the region. The fourth user interface (1540) is a diagram illustrating an example in which some mathematical problems (e.g., ①, ②, ③, ④) are selected by the user.

[0175] FIGS. 16a to 16d are drawings illustrating an example of using an electronic device to solve a mathematical problem according to one embodiment.

[0176] Referring to FIGS. 16a and 16b, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may automatically change the size of an area corresponding to a single mathematical problem according to the input length of the mathematical problem solution process. For example, the processor (120) may provide a first user interface (1610) by taking into account the device characteristics of the electronic device (101) and the user visibility of each mathematical problem. When providing a solution process (1621) corresponding to the first mathematical problem, the processor (120) may enlarge (or increase) the size of the area corresponding to the first mathematical problem, as in the second user interface (1620). It can be seen that the position of the first area (e.g., dotted line (1611)) for the first mathematical problem in the first user interface (1610) is different from the position of the second area (e.g., dotted line (1622)) for the first mathematical problem in the second user interface (1620).

[0177] Referring to FIG. 16c, the third user interface (1630) illustrates an example of changing a handwriting area (e.g., an empty area) based on user input (1631). The processor (120) can change the handwriting area based on user input (1631) that moves the first baseline (1633) to the second baseline (3635).

[0178] Referring to FIG. 16d, the fourth user interface (1640) illustrates an example of displaying a problem across the entire screen based on the screen size of the electronic device (101). When a solution process object (1641) is selected in the fourth user interface (1640), the processor (120) may provide a fifth user interface (1650). The fifth user interface (1650) illustrates an example of displaying a solution process (1651) over a mathematical problem as a pop-up window. The processor (120) may display a solution process (1651) over a mathematical problem as in the fifth user interface (1650) based on the screen size of the electronic device (101).

[0179] FIGS. 17a to 17c are drawings illustrating an example of using an electronic device to solve a mathematical problem according to one embodiment.

[0180] Referring to FIG. 17a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may display a first user interface (1710) on a display (e.g., display module (160) of FIG. 1) based on the screen size of the electronic device (101). The first user interface (1710) illustrates an example in which a mathematical problem (1711) is displayed on the left side of the screen and a blank area (1713) on the right side of the screen where the user can input handwriting. The processor (120) may analyze the handwriting entered in the blank area (1713) to determine if there is an error in the solution process. If there is an error in the solution process, the processor (120) may display a correction object (1715) (e.g., icon, image) on the error (e.g., B(0.2)).

[0181] Referring to FIG. 17b, the processor (120) can display a second user interface (1730) on the display module (160) when a correction object (1715) is selected in the first user interface (1710). The second user interface (1730) illustrates an example in which a replacement variable (1731) for correcting an error in the solution process (e.g., B (0.2)) is displayed as a pop-up window. The processor (120) can distinguish (e.g., highlight) the content corrected in the replacement variable (1731).

[0182] Referring to FIG. 17c, the processor (120) can display a third user interface (1750) on the display module (160) when a replacement variable (1731) is selected in the second user interface (1730). The third user interface (1750) illustrates an example in which B (0.2) is changed to B (0.1) (1751) during the solution process.

[0183] FIGS. 18a to 18c are drawings illustrating an example of providing AI mathematical functions related to problem-solving in an electronic device according to one embodiment.

[0184] Referring to FIG. 18a, the first user interface (1810) may represent an AI mathematical function related to problem solving. A processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may recognize a mathematical problem and provide a problem-solving process (1811).

[0185] Referring to FIG. 18b, when the solution process hide object (1813) is selected, the processor (120) can display a second user interface (1820) on a display (e.g., the display module (160) of FIG. 1). When the handwriting (solution process) hide object (1813) is selected in the first user interface (1810), the processor (120) can display a second user interface (1820) that hides the handwriting by toggling from an eye shape (e.g., the handwriting (solution process) hide object (1813)) to a closed eye shape (e.g., the hide object (1821)). When the hide object (1821) is selected again in the second user interface (1820), it can be changed back to the first user interface (1810) that shows the hidden handwriting (solution process). When comparing the first user interface (1810) and the second user interface (1820), it can be seen that the first user interface (1810) includes the problem-solving process (1811) for the first problem, whereas the second user interface (1820) has the first problem-solving process (1811) removed (or deleted). When the processor (120) selects the hide object (1821), it may hide only the solution process for the selected problem. Alternatively, when the processor (120) selects the hide object (1821), it may hide the handwritten solution process for the entire problem displayed on the display module (160). The user can try solving the first math problem again using the solution process hide function.

[0186] Referring to FIG. 18c, the third user interface (1830) may represent an AI math function related to problem solving. When the AI ​​math function (1831) related to the problem is selected, the processor (120) may provide a pop-up window (1833). The pop-up window (1833) may be provided as a guide UI for importance and correct answer when a pen hover input is made over the importance and correct answer buttons. The importance can be changed to a 3-star level each time the corresponding button is tapped, and the correct answer button can be changed to a 2-star level between correct and incorrect answers each time it is tapped. Alternatively, the processor (120) may recognize the user's handwriting over the problem and automatically input the importance or correct answer.

[0187] FIGS. 19a to 19f are drawings illustrating an example of controlling an AI mathematical function in an electronic device according to one embodiment.

[0188] Referring to FIG. 19a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may display a first user interface (1910) on a display (e.g., display module (160) of FIG. 1) when a math helper object (1911) of a specified application (e.g., note application) is selected. The first user interface (1910) illustrates an example in which a guide message (1913) for entering an AI math function is displayed.

[0189] Referring to FIG. 19b, the second user interface (1920) may include an input area (1921) (or math box) for an AI math function on the execution screen of a specified application. For example, the processor (120) may display the input area (1921) for an AI math function at the location where the tab input was entered when a tab input is entered by a user into an empty area. The size of the input area (1921) for an AI math function may be determined by a default value. The default value may be determined differently depending on the size of the display area of ​​the display module (160). The processor (120) may create the input area (1921) for an AI math function with a specified (e.g., default) size on the execution screen of a specified application (e.g., a blank note). If there is no handwriting entered within the execution screen of a specified application and there is an empty area, the processor (120) may create the input area (1921) for an AI math function with a specified size on the current screen. If there is no handwriting entered within the execution screen of a specified application and there is no empty area, the processor (120) moves to the bottom of a certain area and can create an input area (1921) for an AI math function of a specified size.

[0190] Referring to FIG. 19c, a third user interface (1930) illustrates an example of receiving user input (1931) to change the input area of ​​an AI math function. The user can tap and drag the border (or diagonal border) of the input area of ​​the AI ​​math function. The processor (120) can change the position or size of the input area of ​​the AI ​​math function based on the user input (1931).

[0191] Referring to FIG. 19d, the fourth user interface (1940) illustrates an example of setting an input area (1943) for an AI math function based on user input (1941) detected in an empty area. The processor (120) can detect a touch trajectory entered in the form of a lasso in an empty area on the execution screen of a specified application, and recognize the detected touch trajectory as user input (1941) to set and display an input area (1943) for an AI math function.

[0192] Referring to FIG. 19e, the fifth user interface (1950) illustrates an example of setting an input area (1955) for an AI math function based on user input (1953) including handwriting (1951). The processor (120) can detect a touch trajectory that is input in the form of a lasso surrounding an area where handwriting (1951) is input on the execution screen of a designated application, and recognize the detected touch trajectory as user input (1953) to set and display an input area (1955) for an AI math function. At this time, the content of the handwriting (1951) may be included as is in the input area (1955) for the AI ​​math function. In the input area (1955) for the AI ​​math function, a calculation result (e.g., 27500) for a mathematical formula corresponding to the handwriting (1951) may also be provided. The processor (120) can convert and display handwriting (1951) contained in the input area (1955) of the AI ​​math function into digital characters.

[0193] Referring to FIG. 19f, the sixth user interface (1960) illustrates an example of setting an input area (1965) for an AI math function based on user input (1963) including handwriting (1961). The processor (120) can set and display an input area (1965) for an AI math function by recognizing the user input (1963) as drawing a straight line of handwriting (1961) on the execution screen of a specified application. At this time, the content of the handwriting (1961) may be included as is in the input area (1965) for the AI ​​math function. If there is other handwriting (formula) around the straight line area (e.g., within n pixels up / down / left / right), the processor (120) may include the other handwriting in the input area (1965) for the AI ​​math function and provide the calculation result of the other handwriting. The processor (120) can convert and display handwriting (1961) contained in the input area (1965) of the AI ​​math function into digital characters.

[0194] FIGS. 20a to 20c are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0195] Referring to FIG. 20a, the first user interface (2010) illustrates an example of receiving user input (2013) outside the input area (2011) of the AI ​​math function. A processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may detect the last user input (2013) outside the input area (2011) of the AI ​​math function while the AI ​​math function is activated (e.g., focused on the input area (2011) of the AI ​​math function). The state in which the AI ​​math function is activated may be when the last user input is detected in the input area (2011) of the AI ​​math function, such as while entering a formula.

[0196] Referring to FIG. 20b, the processor (120) can display a second user interface (2030) on a display (e.g., the display module (160) of FIG. 1) based on user input (2013). The second user interface (2030) can indicate that the AI ​​math function is in a standby state and that the input area (2031) of the AI ​​math function is not focused.

[0197] Referring to FIG. 20c, the processor (120) may display the second user interface (2030) for a specified time (e.g., 1 second, 2 seconds) and then display the third user interface (2050) on the display module (160). The third user interface (2050) may be in a state where the AI ​​math function is disabled, and the input area (2051) of the AI ​​math function may not be displayed. The input area (2051) of the AI ​​math function may refer to a square box. The processor (120) may indicate that the AI ​​math function is disabled by not visually displaying the shape of the input area (2051) of the AI ​​math function. Alternatively, the processor (120) may change the size (e.g., shrink, reduce) of the input area (2051) of the AI ​​math function to minimize the impact on user input occurring outside the input area (2051) of the AI ​​math function. For example, the processor (120) can reduce the size of the input area (2051) of the AI ​​math function so that the area where user input is detected does not overlap with the input area (2051) of the AI ​​math function. When the input area (2051) of the AI ​​math function is in a deactivated state, and input such as touch or pen hovering occurs within the input area (2051) of the AI ​​math function, the processor (120) can activate and display the input area (2051) of the AI ​​math function.

[0198] FIGS. 21a to 21c are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0199] Referring to FIG. 21a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may determine whether there is content that can be provided in response to an input mathematical expression (2111). For example, if the processor (120) can provide a graph in response to the input mathematical expression (2111), it may display a first user interface (2110) including a visual object (2113) for providing the graph on a display (e.g., display module (160) of FIG. 1). The processor (120) may generate a new graph whenever the Apply button (2115) is selected on the first user interface (2110). The visual object (2113) for providing the graph may be provided when a radian / degree option is required, such as inputting a trigonometric function. Radians / degrees are automatically determined by defined logic and can be changed by tapping the visual object (2113). If only numbers, degrees, or pi are present in the trigonometric function input, the current setting value may be displayed. Tapping the visual object (2113) changes it to radians, and tapping the visual object (2113) again changes it to degrees.

[0200] Referring to FIG. 21b, if the processor (120) can provide a graph corresponding to an input mathematical expression (2121), it can display a second user interface (2120) including a visual object (2123) for providing a graph on a display module (160). When the object (2123) for providing a graph is selected, the processor (120) can create and provide one graph (2125). When the graph (2125) is created, the object (2123) for providing the graph disappears, and if the previously created graph (2125) is deleted, the object (2123) for providing the graph can reappear.

[0201] Referring to FIG. 21c, the third user interface (2130) illustrates an example of providing a graph (2135) below an AI mathematical expression input area (2131). The AI ​​mathematical expression input area (2131) contains multiple mathematical expressions, and when a mathematical expression is changed, the changes can be reflected in the graph (2135) in real time. The processor (120) can provide an object for re-centering when zooming in / out / moving the graph in the third user interface (2130). The priority of the graph creation location may be the bottom, right, left, and next page of the AI ​​mathematical expression input area (2131). A new graph may be created whenever the Apply button (2133) is selected. If the created graph is not displayed on the same screen as the AI ​​mathematical expression input area (2131), a screen movement button may be provided to zoom in / out to that area so that they are displayed together. After the screen movement button is selected and activated, the screen may be zoomed in / out / moved by user input (e.g., multi-touch). The fourth user interface (2140) may illustrate an example in which both a mathematical formula and a graph are selected (2141) according to user input. The processor (120) may provide a fourth user interface (2140) in which both the mathematical formula and the graph included in the AI ​​mathematical formula input area are selected when the Apply button (2133) is selected in the third user interface (2130). Depending on the user selection, the mathematical formula or the graph (2141) may be converted into an image, and when the image is converted, the mathematical formula and the image may be saved as separate images.

[0202] Referring to FIG. 21d, the fifth user interface (2150) illustrates an example in which only a graph is selected. When the first object (2151) is selected in the fifth user interface (2150), the processor (120) may provide a sixth user interface (2160). The sixth user interface (2160) may include a graph (2163) and graph information (2161). The graph information (2161) may include a mathematical expression included in an AI mathematical expression input area, and the color of the graph corresponding to the mathematical expression. The processor (120) may change the color of the graph corresponding to the mathematical expression according to user input.

[0203] Referring to FIG. 21e, the seventh user interface (2170) illustrates an example of selecting one graph (2171) among a plurality of graphs (2171, 2173, 2175). When the graph (2171) is selected (e.g., long press), the processor (120) may display the selected graph (2171) to distinguish it from the unselected graphs (2173, 2175) and provide a menu for editing or deleting the selected graph (2171). The eighth user interface (2180) illustrates an example of highlighting the selected graph (2171) and providing the mathematical expression (y=2x) of the selected graph (2171), the color of the mathematical expression, and an item (2181) for deletion. When the delete button is selected, the processor (120) may provide a ninth user interface (2190) in which the selected graph is deleted. The ninth user interface (2190) may include a first graph (2191) and a second graph (2193) in which the selected graph (2171) has been deleted. In the ninth user interface (2190), graph information (2161) may include graph information in which a mathematical expression for the deleted graph has been deselected.

[0204] FIGS. 22a to 22d are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0205] Referring to FIG. 22a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment receives a first user input including a mathematical expression (2211) and can receive a second user input (2213) in the form of a lasso surrounding the mathematical expression (2211). The first user interface (2210) illustrates an example of receiving the second user input (2213) in the form of a lasso around the mathematical expression (2211).

[0206] Referring to FIG. 22b, the processor (120) may display a second user interface (2220) on a display (e.g., the display module (160) of FIG. 1) based on a second user input (2213). The second user interface (2220) may include a menu item (2221) for editing (e.g., cut, copy, delete, change style) or converting (e.g., convert to text, convert to math) a mathematical expression.

[0207] Referring to FIG. 22c, the processor (120) can display a third user interface (2230) on the display module (160) when a mathematical expression (2231) conversion (e.g., convert to math) is selected. The third user interface (2230) may include a popup window (2223) containing a menu item (2233) that checks whether to apply the conversion to digital characters and converts the handwritten mathematical expression into digital characters.

[0208] Referring to FIG. 22d, the processor (120) can display a fourth user interface (2240) on the display module (160) when a conversion to digital characters is requested. The fourth user interface (2240) may illustrate an example of a handwritten mathematical expression converted into digital characters.

[0209] FIGS. 23a and FIGS. 23b are drawings illustrating an example of modifying a mathematical expression in an electronic device according to one embodiment.

[0210] Referring to FIG. 23a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may provide a mathematical expression having various variables when the input mathematical expression (2301) is unclear. Referring to the first reference numeral (2310), the processor (120) may provide a first mathematical expression (2311), a second mathematical expression (2323), a third mathematical expression (2315), or a fourth mathematical expression (2317) based on user input (e.g., swipe). Alternatively, the processor (120) may provide an inputtable variable object (2330) based on user input (2320) for selecting a variable of the mathematical expression, and may change the variable of the mathematical expression (2340) according to the user's selection in the inputtable variable object (2330).

[0211] Referring to FIG. 23b, the processor (120) may provide a first inputtable variable object (2360) based on user input (2350) selecting a variable of a mathematical expression. When user input (e.g., a swipe) is detected within the first inputtable variable object (2360), the processor (120) may provide a second inputtable variable object (2370) in which the variable is changed. The processor (120) may change the variable of the mathematical expression (2380) in the second inputtable variable object (2370) according to the user's selection.

[0212] FIGS. 24a to 24h are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0213] Referring to FIG. 24a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may provide a result value of a mathematical expression based on user input for handwriting containing a mathematical expression. Referring to the first user interface (2410), when the processor (120) receives a user input (2411) in the form of a lasso surrounding a mathematical expression in the handwriting, it may create an input area (2413) that provides an AI mathematical function. Additionally, the processor (120) may provide a result value of a mathematical expression (e.g., 31300) to the input area (2413) that provides an AI mathematical function. The processor (120) may provide a result value of a mathematical expression based on the pen type, size, or thickness of the input handwriting.

[0214] Referring to FIG. 24b, the processor (120) can provide a result value of a mathematical expression based on user input for handwriting containing a mathematical expression. Referring to the second user interface (2420), when the processor (120) receives user input (2421) of drawing a line over a mathematical expression in handwriting, it can create an input area (2423) that provides an AI mathematical function and provide a result value of a mathematical expression (e.g., 31300) to the input area (2423) that provides an AI mathematical function.

[0215] Referring to FIG. 24c, the third user interface (2430) illustrates an example in which an input area (2433) provides AI mathematical functions based on a lasso-shaped user input (2431) in an empty area where no handwriting is entered.

[0216] Referring to FIG. 24d, the fourth user interface (2440) illustrates an example of changing the size of an input area (2443) that provides AI mathematical functions based on user input (2441) for the input area (2443) that provides AI mathematical functions. The user input (2441) may be a drag input while the size change handler of the input area (2443) that provides AI mathematical functions is selected.

[0217] Referring to FIG. 24e, the processor (120) can generate an input area that provides AI mathematical functions when an empty area of ​​the execution screen of the note application is selected. The fifth user interface (2450) illustrates an example of displaying an input area (2451) that provides AI mathematical functions in an empty area of ​​the execution screen of the note application.

[0218] Referring to FIG. 24f, the processor (120) may receive additional handwriting (2461) outside the input area (2463) providing the AI ​​math function while using the AI ​​math function. The sixth user interface (2460) illustrates an example of receiving additional handwriting (2461) outside the input area (2463) providing the AI ​​math function.

[0219] Referring to FIG. 24g, the processor (120) can disable the input area (2471) of the AI ​​math function based on additional handwriting (2461). The seventh user interface (2470) illustrates an example in which the input area (2471) of the AI ​​math function is disabled.

[0220] Referring to FIG. 24h, the processor (120) can remove the input area of ​​the AI ​​math function as the input area of ​​the AI ​​math function is disabled. The eighth user interface (2480) illustrates an example in which the input area of ​​the AI ​​math function is removed.

[0221] According to one embodiment, as one progresses from the 7th user interface (2470) to the 8th user interface (2480), the input area of ​​the AI ​​math function can be naturally deactivated step by step with visual effects from the point where it loses focus.

[0222] FIGS. 25a to 25d are drawings illustrating an example of utilizing AI mathematical functions in an electronic device according to one embodiment.

[0223] Referring to FIG. 25a, the first user interface (2510) may indicate a state in which the AI ​​math function is disabled for handwriting containing a mathematical formula (2511).

[0224] Referring to FIG. 25b, the second user interface (2530) may illustrate an example of receiving additional handwriting (2531) (e.g., input for modifying the mathematical expression) for a mathematical expression while the AI ​​mathematical function is disabled.

[0225] Referring to FIG. 25c, the processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may activate an AI math function upon additional handwriting (2531) input. The third user interface (2550) may illustrate an example in which the AI ​​math function is activated and an input area (2551) for the AI ​​math function is displayed.

[0226] Referring to FIG. 25d, the processor (120) may provide a result value by recalculating the mathematical expression according to the modification of the mathematical expression. The fourth user interface (2570) may illustrate an example in which the result value (2771) is modified by recalculating the modified mathematical expression.

[0227] FIG. 26 is a drawing illustrating an example of providing a result value for a mathematical formula in an electronic device according to one embodiment.

[0228] Referring to FIG. 26, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may provide a result value based on a specified user input after inputting a mathematical expression. The first user interface (2610) illustrates an example in which, when a specified handwriting (2611) (e.g., =) is input among the mathematical expressions, an execution initiation object (2613) for activating an AI mathematical function is provided. When the execution initiation object (2613) is selected, the processor (120) may activate the AI ​​mathematical function to calculate the mathematical expression and provide a result value. The second user interface (2630) may illustrate an example in which a result value (2631) of the mathematical expression is provided and additional handwriting (2633) is input. The result value (2631) may be provided in a form similar to the handwriting based on the type, size, and thickness of the handwriting pen.

[0229] When the AI ​​math function is activated, the processor (120) can receive a designated handwriting input or calculate an input mathematical expression without a user request and provide a result value. For example, the processor (120) can calculate a mathematical expression and provide a result value if no user input is detected for a designated time after receiving additional handwriting (2633). The third user interface (2650) may be illustrated as providing a result value (2651) of the mathematical expression in response to additional handwriting (2633). Referring to reference numeral (2670), the processor (120) can display the calculated value of the current mathematical expression in real time even if a result request operator (e.g., =) is not entered. If an equal sign is not entered, the processor (120) can provide an equal sign and a calculated result value to the right of the mathematical expression, leaving a margin value in a certain area for additional formula input. If an equal sign is entered, the processor (120) can provide a calculated result value of the mathematical expression to the right of the equal sign.

[0230] FIG. 27 is a diagram illustrating an example of modifying a result value by modifying a mathematical formula in an electronic device according to one embodiment.

[0231] Referring to FIG. 27, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may modify a result value by modifying a mathematical expression. A first user interface (2710) illustrates an example in which a result value (2713) is modified according to a handwriting (2711) that erases and re-enters a value for a variable a. Alternatively, a second user interface (2730) illustrates an example in which a result value (2733) is modified according to a handwriting (2731) (e.g., +5) that inserts a formula in the middle of a mathematical expression. A third user interface (2750) illustrates an example in which a result value (2753) is modified according to a handwriting (2751) that inserts a formula in the middle of a mathematical expression.

[0232] FIG. 28 is a drawing illustrating an example in which a result value is provided differently in an electronic device according to one embodiment.

[0233] Referring to FIG. 28, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may display a first user interface (2810) for setting AI mathematical functions on a display (e.g., display module (160) of FIG. 1). The first user interface (2810) may include menu items (2811), such as converting to text or applying matching colors. A second user interface (2830) illustrates an example of converting a mathematical formula (2831) into digital characters and providing a result value (2833) for the mathematical formula (2831). A third user interface (2850) illustrates an example of applying matching colors to handwriting and providing a result value (2851). The processor (120) can apply the most frequently used handwriting color from notes stored in memory (e.g., memory (130) of FIG. 1) or a cloud server (e.g., server (108) of FIG. 1) as a matching color. Alternatively, if there is no input handwriting or only handwriting of the same color as the formula, the processor (120) can apply a matching color with a specified color (e.g., default color). Alternatively, if there is no input handwriting or only handwriting of the same color as the formula and there is an attachment such as an image, the processor (120) can recommend a color that matches the attachment. The fourth user interface (2870) illustrates an example in which the color of the result value (2871) is changed by applying a matching color to the handwriting.

[0234] FIG. 29 is a diagram illustrating an example of entering an AI mathematical function in an electronic device according to one embodiment.

[0235] Referring to FIG. 29, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may enter an AI math function when a designated object is selected in a calculator application. A first user interface (2910) is an execution screen of a calculator application and illustrates an example in which a designated object (2911) is selected to enter an AI math function. When the designated object (2911) is selected, the processor (120) may display a second user interface (2930) on a display (e.g., display module (160) of FIG. 1). A second user interface (2930) is an execution screen of a note application and illustrates an example in which an AI math function is executed. When a designated object (2911) is selected while the formula 1+1 is being entered in the calculator application, the processor (120) can convert the formula being entered into handwriting and display it on the execution screen of the note application, and the AI ​​math function can be executed to provide the result of the formula. The processor (120) can provide a menu item (2931) related to the AI ​​math function. The third user interface (2950) illustrates an example of providing a separate folder for the handwriting math formula calculation history (2951).

[0236] FIGS. 30a and FIGS. 30b are drawings illustrating an example of setting values ​​for converting handwriting into digital characters in an electronic device according to one embodiment.

[0237] Referring to FIG. 30a, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to one embodiment may provide a user experience for creating notes that set areas. The first user interface (3010) illustrates an example in which a first area (3011) for text input and a second area (3013) for mathematical expression input are designated within the execution screen of a note application. The processor (120) may provide menus for mathematical expression conversion, calculation result color setting, box background design, and application of units to mathematical expressions.

[0238] Referring to FIG. 30b, the second user interface (3030) illustrates an example of providing a menu (3031) for converting mathematical expressions (e.g., converting to text), setting the color of the calculation result (e.g., applying a matching color), or designing a box background and applying units of mathematical expressions.

[0239] A method of operation of an electronic device (101) according to one embodiment of the present disclosure may include: receiving handwriting containing a mathematical formula; converting the handwriting into digital characters and displaying them on a display (160) of the electronic device; if no additional handwriting is entered by a user for a specified period of time, calculating and displaying a result value corresponding to the mathematical formula; receiving user input on the entered mathematical formula; modifying the mathematical formula based on the user input; and recalculating and displaying a result value corresponding to the modified mathematical formula.

[0240] The above method may further include receiving a second user input for selecting a variable within the above mathematical formula, providing a variable object that can be input corresponding to the selected variable, and reflecting the selected variable from the provided variable object into the above mathematical formula.

[0241] The above method may further include the operation of determining mathematical symbols or characters corresponding to the handwriting by analyzing notes stored in the memory or an external server connected to the electronic device.

[0242] The above-determining operation may include an operation to determine the user's math level by analyzing the stored note, and an operation to determine a mathematical symbol or character corresponding to the handwriting based on the determined math level and the content of the analyzed note.

[0243] The above-determining operation may include an operation of determining whether there is a mathematical symbol or character corresponding to the input mathematical formula among the stored notes, and an operation of determining a mathematical symbol or character corresponding to the handwriting based on the corresponding note if there is a mathematical symbol or character corresponding to the handwriting among the stored notes.

[0244] The above-determining operation may include: obtaining the age of the user based on a user account set in the electronic device; identifying a mathematical learning stage by analyzing a mathematical expression included in the stored note; identifying the frequency of use of a mathematical expression by comparing the mathematical expression in the stored note with the mathematical expression included in the handwriting; determining grammatical errors based on the positional relationship of the mathematical expression included in the handwriting; and determining the mathematical level of the user based on at least one of the age, the mathematical learning stage, the frequency of use, or the grammatical errors.

[0245] The above method may include the operation of determining a user's mathematical level by analyzing a note stored in the memory or a server connected to the electronic device, the operation of providing a result value corresponding to the mathematical formula based on the first mathematical level, the operation of providing a summary solution process and result value corresponding to the mathematical formula based on the second mathematical level, and the operation of providing a detailed solution process and result value corresponding to the mathematical formula based on the third mathematical level.

[0246] The above method may further include an action of providing a visual object for changing the level of summary of the solution process, and an action of changing and providing the level of summary of the solution process according to the user's selection regarding the visual object.

[0247] The above method may further include the operation of obtaining a problem and a user's answer through content scanning or content loading, the operation of recognizing a problem by deleting the user's answer from the obtained problem, and the operation of generating the recognized problem as an incorrect answer note based on the user's selection.

[0248] The various embodiments of the present invention disclosed in this specification and drawings are provided merely as specific examples to facilitate the explanation of the technical content of the invention and to aid in understanding the invention, and are not intended to limit the scope of the invention. Accordingly, the scope of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the invention, in addition to the embodiments disclosed herein.

Claims

1. In an electronic device (101), Display (160), Memory (130) for storing instructions; and The electronic device includes a processor (120), and when the instructions are executed by the processor, the electronic device Receive handwritten text containing mathematical expressions, and Converts the above handwriting into digital characters and displays it, If no additional handwriting is entered by the user for a specified period, calculate and display a result value corresponding to the above mathematical formula, and Receive user input on the above-mentioned mathematical formula, and Modify the above mathematical formula based on the above user input, and An electronic device that recalculates and displays a result value corresponding to the above modified mathematical formula.

2. In paragraph 1, when the instructions are executed by the processor, the electronic device, An electronic device that displays an operator together with a result value corresponding to the mathematical expression when no additional handwriting is entered by the user for a specified period of time and no result request operator is entered by the user.

3. In paragraph 1, when the instructions are executed by the processor, the electronic device, Receiving user input to select a variable within the above mathematical formula, and Provides an inputable variable object corresponding to the above-mentioned selected variable, and An electronic device that reflects a selected variable from the provided variable object into the mathematical formula.

4. In paragraph 1, when the instructions are executed by the processor, the electronic device, An electronic device that analyzes notes stored in the memory or an external server connected to the electronic device to determine mathematical symbols or characters corresponding to the handwriting.

5. In paragraph 4, when the instructions are executed by the processor, the electronic device, Analyze the above-mentioned saved notes to determine the user's math level, and An electronic device for determining a mathematical symbol or character corresponding to the handwriting based on the above-determined mathematical level and the content of the above-determined analyzed note.

6. In paragraph 4, when the instructions are executed by the processor, the electronic device, Determining whether there is a mathematical symbol or character corresponding to the input mathematical formula among the above-mentioned stored notes, and An electronic device that determines a mathematical symbol or character corresponding to the handwriting based on the said note when a mathematical symbol or character corresponding to the said handwriting exists among the said stored notes.

7. In paragraph 4, when the instructions are executed by the processor, the electronic device, Based on the user account set in the electronic device, the age of the user is obtained, and Identify the math learning stages by analyzing the mathematical formulas included in the above-mentioned saved notes, and Identify the frequency of use of mathematical formulas by comparing the mathematical formulas in the above-saved notes with the mathematical formulas included in the above-saved handwriting, and Judging grammatical errors based on the positional relationship of mathematical expressions included in the above handwriting, and An electronic device for determining the user's math level based on at least one of the above age, the above math learning stage, the above usage frequency, or the above grammatical errors.

8. In paragraph 1, when the instructions are executed by the processor, the electronic device, Determining the user's math level by analyzing notes stored in the memory or a server connected to the electronic device, and Based on the first level of mathematics, a result value corresponding to the above mathematical formula is provided, and Based on the second level of mathematics, a summary solution process and result value corresponding to the above mathematical formula are provided, and An electronic device that provides a detailed solution process and result value corresponding to the above mathematical formula based on a third level of mathematics.

9. In paragraph 8, when the instructions are executed by the processor, the electronic device, It provides visual objects to change the level of summary of the solution process, and An electronic device that changes and provides the level of summary of the solution process according to the user's selection regarding the above visual object.

10. In paragraph 1, when the instructions are executed by the processor, the electronic device, An electronic device that provides a solution process or result value on the problem when a problem corresponding to a handwritten text containing the above mathematical formula is obtained.

11. In paragraph 1, when the instructions are executed by the processor, the electronic device, Acquire questions and user answers through content scanning or content loading, and Recognize the problem by deleting the user's answer from the problem obtained above, and An electronic device that generates the above-mentioned recognized problem as an incorrect answer note based on the user's selection.

12. In paragraph 11, when the instructions are executed by the processor, the electronic device, An electronic device that adjusts a layout providing a problem and a solution process corresponding to the problem based on the display size of the electronic device.

13. In the method of operating the electronic device (101), Action of receiving handwriting containing mathematical expressions; The operation of converting the above handwriting into digital characters and displaying them on the display (160) of the electronic device; An operation to calculate and display a result value corresponding to the above mathematical formula if no additional handwriting is entered by the user for a specified period; The operation of receiving user input on the above-mentioned mathematical formula; An operation to modify the mathematical formula based on the above user input; and A method including the operation of recalculating and displaying a result value corresponding to the above modified mathematical formula.

14. In Clause 13, the operation of calculating and displaying the above result value is, A method including the action of displaying an operator together with a result value corresponding to the mathematical expression when no additional handwriting is entered by the user for a specified period of time and no result request operator is entered by the user.

15. In Paragraph 13, The operation of receiving a second user input selecting a variable within the above mathematical formula; An operation to provide an inputtable variable object corresponding to the above-mentioned selected variable; and A method further comprising the operation of reflecting a selected variable from the provided variable object into the mathematical formula.

Images