Electronic device including memo function and method of operating the same

Abstract

An electronic device according to embodiments of the present disclosure may include a display module, a processor, and a memory. The display module can acquire a user's handwriting via user input when the screen display is off. The processor can control the operation of the display module. The memory may be operatively connected to the processor and may include instructions. The memory may include a first handwriting memo and a second handwriting memo related to the handwriting acquired when the screen display is off. When the instructions are executed by the processor, the electronic device can generate electronic text from the first and second handwriting memos related to the handwriting acquired when the screen display is off. When the instructions are executed by the processor, the electronic device can store the electronic text in the memory. When the instructions are executed by the processor, the electronic device can simultaneously display the electronic text on the display module along with the first and second handwriting memos. Various other embodiments are also possible.

Description

Technical Field

[0001] This disclosure relates to an electronic device and its operating method, which can convert a handwriting memo in a screen-off state into an electronic text document. Background Technology

[0002] Electronic devices are striving for thinness, lightness, miniaturization, and multifunctionality, and to this end, various components and displays are incorporated into them. With advancements in display technology, research and development of electronic devices, including displays (or flexible displays), is actively underway. Electronic devices may include digitizers for converting analog coordinates (e.g., position) from an electronic pen (e.g., a stylus) into digital data. It is possible to obtain handwriting memos from a user while the screen of the electronic device is off, and to display the obtained screen-off handwriting on the screen (e.g., a screen-off memo).

[0003] The above information is presented as relevant technical information to aid in understanding this disclosure. No determination or assertion is made regarding whether any of the above content can be used as prior art in relation to this disclosure. Summary of the Invention

[0004] Solution to the problem

[0005] Electronic devices can offer screen-off handwriting (e.g., screen-off memos) functionality, allowing users to quickly jot down important notes. Although the number of words included in screen-off handwriting (e.g., screen-off memos) is small, it can describe core keywords. Because screen-off handwriting (e.g., screen-off memos) is done in a short time, it can be written in a less refined and disorganized manner, rather than in a carefully crafted way.

[0006] Embodiments of this disclosure can provide functionality for processing and organizing screen-off handwriting (e.g., screen-off memos) to improve user convenience.

[0007] Embodiments of this disclosure may provide an electronic device and its operating method, which can convert handwritten memos in a screen-off state into electronic text documents and store the electronic text documents in a memory (e.g., a database, a server).

[0008] Embodiments of this disclosure may provide an electronic device and its operating method, which can convert handwritten memos in a screen-off state into electronic text documents and display the electronic text documents on a display.

[0009] Embodiments of this disclosure may provide an electronic device and a method of operating the same, which can convert a screen-off memo into multiple electronic text documents and merge the multiple electronic text documents.

[0010] Embodiments of this disclosure may provide an electronic device and a method of operating the same, which can convert a screen-off memo into multiple electronic text documents and merge the multiple electronic text documents to display them on a display.

[0011] The technical problems to be solved by the present invention are not limited to those described above. Those skilled in the art will clearly understand from the following description other technical problems not mentioned.

[0012] An electronic device according to embodiments of the present disclosure may include a display module, a processor, and a memory. The display module may be configured to acquire a user's handwriting via user input when the screen is off. The processor may be configured to control the operation of the display module. The memory may be operatively connected to the processor and includes instructions. The memory may include a first handwriting memo and a second handwriting memo related to the handwriting acquired when the screen is off. When the instructions are executed by the processor, the electronic device may be configured to generate electronic text from the first and second handwriting memos related to the handwriting acquired when the screen is off. When the instructions are executed by the processor, the electronic device may be configured to store the electronic text in the memory. When the instructions are executed by the processor, the electronic device may be configured to simultaneously display the electronic text along with the first and second handwriting memos on the display module.

[0013] The operation method of the electronic device according to embodiments of the present disclosure can obtain user handwriting input on a display screen while the screen is off. The operation method can convert a first handwriting memo and a second handwriting memo related to the handwriting obtained while the screen is off into electronic text. The operation method can store the electronic text in a memory. The operation method can simultaneously display the electronic text on the display screen along with the first and second handwriting memos.

[0014] In a recording medium storing processor-readable instructions for an electronic device, when the processor executes the instructions, the instructions cause the electronic device to obtain user handwriting input on a display screen while the screen is off. When executed by the processor, the instructions cause the electronic device to convert a first handwriting memo and a second handwriting memo related to the handwriting obtained while the screen is off into electronic text. When executed by the processor, the instructions cause the electronic device to store the electronic text in memory. When executed by the processor, the instructions cause the electronic device to simultaneously display the electronic text along with the first and second handwriting memos on the display screen.

[0015] The electronic device and its operating method according to embodiments of the present disclosure can convert handwritten memos in a screen-off state into electronic text documents and store the electronic text documents in a memory (e.g., a database, a server).

[0016] The electronic device and its operating method according to embodiments of the present disclosure can convert handwritten memos in a screen-off state into electronic text documents and display the electronic text documents on a display.

[0017] The electronic device and its operating method according to embodiments of the present disclosure can convert a screen-off memo into multiple electronic text documents and merge the multiple electronic text documents.

[0018] The electronic device and its operating method according to embodiments of the present disclosure can convert a screen-off memo into multiple electronic text documents and merge the multiple electronic text documents to display them on a display.

[0019] In addition, various effects that can be directly or indirectly identified through this document can be provided. Attached Figure Description

[0020] Regarding the description of the accompanying drawings, the same or similar reference numerals may be used for the same or similar parts.

[0021] Figure 1 This is a block diagram of an electronic device in a network environment according to embodiments of the present disclosure.

[0022] Figure 2 This is a perspective view of a first surface (e.g., front surface) of an electronic device according to an embodiment of the present disclosure.

[0023] Figure 3 This is a perspective view of the second surface (e.g., the rear surface) of an electronic device according to an embodiment of the present disclosure.

[0024] Figure 4 This is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0025] Figure 5 This is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0026] Figure 6 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0027] Figure 7 This is a diagram illustrating the storage of text data (e.g., electronic text documents) in a database.

[0028] Figure 8 This is an illustration showing an example of learning text and handwriting style in the AI ​​module.

[0029] Figure 9 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0030] Figure 10 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0031] Figure 11 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0032] Figure 12 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0033] Figure 13 This is a diagram showing the storage of screen-off handwriting (e.g., screen-off memos) and stroke information obtained from screen-off handwriting in a database.

[0034] Figure 14 This is a diagram showing how to identify words in handwriting when the screen is off (e.g., a screen-off memo).

[0035] Figure 15 This is a diagram showing the lines in a screen-off handwriting (e.g., a screen-off memo).

[0036] Figure 16 This is a diagram showing the line recognition results of handwriting with the screen off (e.g., a memo with the screen off).

[0037] Figure 17 This is a diagram illustrating the post-processing results of an electronic device according to an embodiment of the present disclosure.

[0038] Figure 18 This is a diagram illustrating an example of using prompts for document organization on an electronic device according to an embodiment of the present disclosure.

[0039] Figure 19 This is a diagram illustrating the document organization results of an electronic device according to an embodiment of the present disclosure.

[0040] Figure 20 This is a diagram illustrating the document organization results of an electronic device according to an embodiment of the present disclosure.

[0041] Figure 21 This is a diagram illustrating an example of a document merging prompt using an electronic device according to an embodiment of the present disclosure.

[0042] Figure 22 This is a diagram illustrating the document merging result of an electronic device according to an embodiment of the present disclosure.

[0043] Figure 23This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0044] It should be noted that throughout the accompanying drawings, the same reference numerals are used to depict the same or similar elements, features, and structures. Detailed Implementation

[0045] The following description with reference to the accompanying drawings is intended to aid in a full understanding of the various embodiments of this disclosure as defined by the claims and their equivalents. It includes various specific details to aid understanding, but these are merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and subject matter of this disclosure. Additionally, for clarity and brevity, descriptions of well-known functions and components may be omitted.

[0046] The terms and words used in the following description and claims are not limited to their dictionary meanings, but are used solely by the applicant to ensure a clear and consistent understanding of this disclosure. Therefore, those skilled in the art should understand that the following description of various embodiments of this disclosure is for illustrative purposes only and is not intended to limit the scope of this disclosure as defined by the appended claims and their equivalents.

[0047] It should be understood that, unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the” include the plural referent. Thus, for example, a reference to “component surface” includes a reference to one or more such surfaces.

[0048] Figure 1 This is a block diagram illustrating an electronic device 101 in a network environment 100 according to an embodiment of the present disclosure.

[0049] Reference Figure 1In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, user identification module (SIM) 196, or antenna module 197. In some embodiments, at least one of the above components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some of the components described above (e.g., sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (e.g., display module 160).

[0050] Processor 120 may run software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, processor 120 may load commands or data received from another component (e.g., sensor module 176 or communication module 190) into volatile memory 132, process the commands or data stored in volatile memory 132, and store the resulting data in non-volatile memory 134. According to embodiments, processor 120 may include a main processor 121 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or to be adapted to perform a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or may implement a portion of the main processor 121.

[0051] When the main processor 121 is inactive (e.g., in sleep mode), the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., running an application), the auxiliary processor 123 can, together with the main processor 121, control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to embodiments, the auxiliary processor 123 (e.g., a neural network processing device) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed within the electronic device 101 executing the artificial intelligence model itself or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural networks may be, but are not limited to, deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), or deep Q-networks, or combinations of two or more thereof. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0052] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, software (e.g., program 140) and input or output data for commands associated with it. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0053] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0054] Input module 150 can receive commands or data from outside electronic device 101 (e.g., a user) that will be used by other components of electronic device 101 (e.g., processor 120). Input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0055] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to an embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0056] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display module 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 160 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0057] The audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or a speaker or headphones of an external electronic device (e.g., electronic device 102) that is directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0058] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, an angle sensor, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor (e.g., a geomagnetic sensor), an accelerometer, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0059] Interface 177 may support one or more specific protocols used to enable electronic device 101 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0060] Connection 178 may include a connector, through which electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102). According to embodiments, connection 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0061] The haptic module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0062] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0063] The power management module 188 can manage the power supplied to or consumed by the electronic device 101. According to an embodiment, the power management module 188 can be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0064] Battery 189 can power at least one component of electronic device 101. According to an embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable accumulator, or a fuel cell.

[0065] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors capable of operating independently of processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components (e.g., multiple chips) that are separate from each other. The wireless communication module 192 uses user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196 to identify and verify the electronic device 101 in the communication network (such as the first network 198 or the second network 199).

[0066] Wireless communication module 192 can support 5G networks following 4G networks and next-generation communication technologies (e.g., new radio (NR) access technologies). NR access technologies can support high-speed transmission of large amounts of data (i.e., enhanced mobile broadband (eMBB)), minimized terminal power consumption and connectivity of multiple terminals (massive machine-type communication (mMTC)) or high reliability and low latency (ultra-reliable low-latency communication (URLLC)). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure the performance of high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic devices (e.g., electronic device 104), or network systems (e.g., second network 199). According to an embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0067] Antenna module 197 can transmit or receive signals or power to or from the exterior of electronic device 101 (e.g., external electronic device). According to an embodiment, antenna module 197 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190 (e.g., wireless communication module 192). Signals or power can then be transmitted or received between communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0068] According to an embodiment, antenna module 197 can form a millimeter-wave antenna module. According to an embodiment, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top or side surface) of the printed circuit board and are capable of transmitting or receiving signals of the specified high-frequency band.

[0069] At least some of the aforementioned components can be interconnected via a peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)) and can communicatively transmit signals (e.g., commands or data) between them.

[0070] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 connected to a second network 199. Each of electronic device 102 or electronic device 104 can be a device of the same type as electronic device 101, or a device of a different type. According to an embodiment, all or some operations that would be performed on electronic device 101 can be performed on one or more of external electronic devices 102, external electronic devices 104, or server 108. For example, if electronic device 101 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service, instead of running the function or service, or electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 101. Electronic device 101 may provide the result as at least a partial response to the request, with or without further processing of the result. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to embodiments of this disclosure, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology or IoT-related technologies.

[0071] The electronic device according to embodiments of this disclosure can be one of various types of electronic devices. For example, the electronic device may include a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. However, the electronic device according to embodiments of this disclosure is not limited to any of the above.

[0072] The embodiments disclosed herein and the terminology used herein are not intended to limit the technical features set forth herein to the particular embodiments, and include various changes, equivalents, or substitutions to those embodiments. Regarding the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It should be understood that, unless the relevant context clearly indicates otherwise, the singular form of the noun corresponding to an item may include one or more things. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items listed together in the corresponding phrase. As used herein, terms such as “first” and “second” or “first” and “second” may be used simply to distinguish the respective components from one another and do not limit the components in other respects (e.g., importance or order). If a particular element (e.g., a first element) is referred to as "combined to another element (e.g., a second element)" or "connected to another element (e.g., a second element)" when the terms "operably" or "communically" are used, or when the terms "operably" or "communically" are not used, then the particular element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0073] The term "module" as used in the various embodiments of this disclosure may include a unit implemented in hardware, software, or firmware, or any combination thereof, and may be used interchangeably with other terms such as logic, logic block, component, or circuit. A module may be a single integrated component adapted to perform one or more functions, or its smallest unit or portion. For example, according to an embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0074] Embodiments of this disclosure may be implemented as software (e.g., program 140) containing one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, under the control of a processor, the processor (e.g., processor 120) of the machine (e.g., electronic device 101) may invoke and execute at least one or more instructions stored in the storage medium, with or without one or more other components. This allows the machine to operate to perform at least one function according to the invoked at least one instruction. One or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" means only that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored in the storage medium.

[0075] The methods disclosed in the embodiments of this document can be provided as included in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product can be distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or via an app store (e.g., the Play Store). TM This can be done online (e.g., downloading or uploading) between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

[0076] According to embodiments, each of the above components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately located in other components. According to embodiments, one or more of the above components or operations may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as the corresponding components in the multiple components before integration. According to various embodiments, operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more operations may be performed in a different order or omitted, or one or more other operations may be added.

[0077] According to an embodiment, the display module 160 may include a bar-type or panel-type display (e.g., Figure 2 Monitor 201 in Figure 4 The display 410 in the display driver IC (e.g., Figure 4 The display driver IC 430 in the display circuit (e.g., touch circuit) Figure 4 The touch circuit 450 in the middle), digitizer (e.g., Figure 2 and Figure 4 Digitizer 460) and digitizer driver (e.g., Figure 4 The digitizer driver 470 in the middle.

[0078] According to an embodiment, the electronic device 101 may include a display module 160 and an electronic pen (e.g., a stylus) (e.g., Figure 2 and Figure 3 (e.g., electronic pen 300). For example, display module 160 may include a flexible display configured to fold or unfold. For example, display module 160 may include a display (e.g., electronic pen 300). Figure 4 The display 410 in the display driver IC (e.g., Figure 4 The display driver IC 430 in the display circuit (e.g., touch circuit) Figure 4 The touch circuit 450 in the middle), digitizer (e.g., Figure 2 and Figure 4 Digitizer 460) and digitizer driver (e.g., Figure 4 The digitizer driver 470 in the middle.

[0079] According to an embodiment, the display module 160 may include a flexible display slidably disposed to provide a screen (e.g., a display screen), and a display driver IC (e.g., Figure 4 The display driver IC 430 in the display circuit (e.g., touch circuit) Figure 4 The touch circuit 450 in the middle), digitizer (e.g., Figure 2 and Figure 4 Digitizer 460) and digitizer driver (e.g., Figure 4 The digitizer driver 470 in the middle.

[0080] According to an embodiment, the display module 160 may also be referred to as a deformable display (e.g., a stretchable display), an expandable display, or a slide-out display.

[0081] Figure 2 This is a perspective view of a first surface (e.g., front surface) of an electronic device according to an embodiment of the present disclosure. Figure 3 This is a perspective view of the second surface (e.g., the rear surface) of an electronic device according to an embodiment of the present disclosure.

[0082] Reference Figure 2 and Figure 3 Electronic device 200 according to embodiments of the present disclosure (e.g., Figure 1 The electronic device 101 may include a first surface (or front surface) 210A, a second surface (or rear surface) 210B and a housing 210.

[0083] Electronic device 200 according to embodiments of the present disclosure (e.g., Figure 1 The electronic device 101 may include a display 201, a display driver IC (e.g., Figure 4 The display driver IC 430 in the display circuit (e.g., touch circuit) Figure 4 Touch circuit 450), digitizer 460 (e.g., Figure 4 Digitizer 460 in the middle), digitizer driver (e.g., Figure 4 The digitizer driver 470 and the electronic pen 300 (e.g., stylus) are included.

[0084] According to an embodiment, the display 201 may be supported by the housing 210. For example, the display 201 may include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, or a micro LED display.

[0085] According to an embodiment, the housing 210 may include a side surface 210C surrounding the space between the first surface 210A and the second surface 210B. According to an embodiment, the housing 210 may refer to a structure forming portions of the first surface 210A, the second surface 210B, and the side surface 210C.

[0086] According to an embodiment, the first surface 210A may be formed of a front panel 202 (e.g., a glass or polymer panel including various coatings), at least a portion of which is substantially transparent.

[0087] According to an embodiment, the second surface 210B may be formed of a substantially opaque back plate 211. The back plate 211 may be formed of, for example, coated or colored glass, ceramic, polymer, metal (e.g., aluminum, stainless steel (STS), or magnesium), or a combination of at least two of the above materials. However, this disclosure is not limited thereto, and the back plate 211 may be formed of transparent glass.

[0088] According to an embodiment, the side surface 210C is connected to the front panel 202 and the rear panel 211, and may be formed of a side frame structure 218 (or “side member”) (or “side frame”) comprising metal and / or polymer.

[0089] According to an embodiment, the back panel 211 and the side frame structure 218 may be integrally formed and comprise the same material (e.g., a metallic material such as aluminum).

[0090] According to an embodiment, the front panel 202 may include two first regions 210D that curve and extend seamlessly from the first surface 210A toward the rear panel 211. The two first regions 210D may be located at two long edges of the front panel 202.

[0091] According to an embodiment, the rear panel 211 may include two second regions 210E that bend and extend seamlessly from the second surface 210B toward the front panel 202.

[0092] According to an embodiment, the front panel 202 (or the rear panel 211) may include only one of the first region 210D (or the second region 210E). According to an embodiment, some of the first region 210D or the second region 210E may not be included.

[0093] In one embodiment, when viewed from the side of the electronic device 200, the side bezel structure 218 may have a first thickness (or width) in the portion excluding the first region 210D or the second region 210E. In another embodiment, when viewed from the side of the electronic device 200, the side bezel structure 218 may have a second thickness (or width) less than the first thickness in the portion including the first region 210D or the second region 210E.

[0094] According to an embodiment, the electronic device 200 may include a display 201 and an audio input device 203 (e.g., Figure 1 The input module 150, microphone), and audio output devices 207 and 214 (e.g., Figure 1 The audio output module 155, speaker (e.g., audio module), and sensor modules 204 and 219 (e.g., Figure 1 Sensor module 176), camera modules 205 and 212 (e.g., Figure 1 The camera module 180, flash 213, key input device 217, indicator (not shown), and connectors 208 and 209 are at least one of them.

[0095] According to an embodiment, the electronic device 200 may omit at least one of the above-mentioned components (e.g., key input device 217), or may include other components.

[0096] According to an embodiment, the display 201 can be seen through the top surface of the front panel 202.

[0097] According to an embodiment, at least a portion of the display 201 is visible through a front panel 202, which forms a first region 210D of a first surface 210A and a side surface 210C. For example, the display 201 may be integrated with touch sensing circuitry, a pressure sensor capable of measuring the intensity (pressure) of a touch, and / or a digitizer (e.g., a magnetic field type electronic pen 300, e.g., a stylus) for detecting magnetic fields. Figure 4 The display 201 may be coupled to the digitizer 460 of the magnetic field-detecting electronic pen 300 (e.g., a stylus). For example, the display 201 may be configured to be adjacent to the digitizer 460 of the magnetic field-detecting electronic pen 300 (e.g., a stylus).

[0098] For example, touch sensors (e.g., Figure 4 The touch sensor 451 can be positioned above the display 201 along the z-axis direction (e.g., on the upper surface of the display 201). For example, the digitizer 460 can be positioned below the display 201 along the z-axis direction (e.g., on the lower surface of the display 201).

[0099] According to an embodiment, at least a portion of the sensor modules 204 and 219 and / or at least a portion of the key input device 217 may be disposed in the first region 210D and / or the second region 210E.

[0100] According to an embodiment, at least one of the first sensor module 204, camera modules 205 and 212 (e.g., an image sensor and an image sensor driving circuit), audio output device 214 (e.g., an audio module), and fingerprint sensor may be included on the rear surface of the screen display area of ​​the display 201.

[0101] According to an embodiment, the display 201 may be coupled to or adjacent to a touch sensing circuit and a pressure sensor capable of measuring the intensity (pressure) of the touch.

[0102] According to an embodiment, the display 210 may be coupled to or arranged adjacent to a digitizer 460 that detects a magnetic field type electronic pen (e.g., a stylus).

[0103] According to an embodiment, at least a portion of the sensor modules 204 and 219 and / or at least a portion of the key input device 217 may be disposed in the first region 210D and / or the second region 210E.

[0104] According to one embodiment, the audio input device 203 may include a microphone. According to another embodiment, the input device 203 may include a plurality of microphones arranged to sense the direction of sound.

[0105] According to an embodiment, audio output devices 207 and 214 may include audio output device 207 operating as an external speaker and audio output device 214 operating as a receiver for making calls.

[0106] In some embodiments, audio input device 203 (e.g., a microphone), audio output devices 207 and 214, and connectors 208 and 209 may be disposed within the internal space of electronic device 200. Audio input device 203 (e.g., a microphone), audio output devices 207 and 214, and connectors 208 and 209 may be exposed to the external environment through at least one aperture formed in housing 210. In some embodiments, the aperture formed in housing 210 may be used collectively for audio input device 203 (e.g., a microphone) and audio output devices 207 and 214. In some embodiments, audio output devices 207 and 214 may include a loudspeaker (e.g., a piezoelectric loudspeaker) that operates without the aperture formed in housing 210.

[0107] According to an embodiment, an electronic pen 300 (e.g., a stylus) may be housed within the internal space of an electronic device 200. The electronic pen 300 may be inserted into and disposed within a portion of the space of the electronic device 200. The electronic pen 300 may be inserted (e.g., pushed in) or removed (e.g., pulled out) through a pen hole (not shown) formed in the side surface of the electronic device 200. When in use, the electronic pen 300 (e.g., a stylus) may be removed (e.g., pulled out) from the interior of the electronic device 200 to the exterior. When not in use, the electronic pen 300 (e.g., a stylus) may be inserted (e.g., pushed in) into the interior of the electronic device 200.

[0108] According to an embodiment, an electronic pen 300 (e.g., a stylus) can be magnetically attached to a surface (e.g., front, rear, or side surface) of an electronic device 200.

[0109] According to an embodiment, sensor modules 204 and 219 (e.g., Figure 1 The sensor module 176 can generate electrical signals or data values ​​corresponding to the internal operating state or external environmental state of the electronic device 200. Sensor modules 204 and 219 may include a first sensor module 204 (e.g., a proximity sensor) disposed on a first surface 210A of the housing 210, and / or a second sensor module 219 (e.g., an HRM sensor) disposed on a second surface 210B of the housing 210, and / or a third sensor module (not shown) (e.g., a fingerprint sensor). For example, a fingerprint sensor may be disposed on the first surface 210A (e.g., the display 201) and / or the second surface 210B of the housing 210.

[0110] The electronic device 200 may also include at least one of a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor (e.g., a geomagnetic sensor), a 6-axis sensor, an accelerometer, a grip sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, and / or an illuminance sensor.

[0111] According to an embodiment, camera modules 205 and 212 may include a first camera module 205 disposed on a first surface 210A of the electronic device 200 and a second camera module 212 disposed on a second surface 210B. A flash 213 may be disposed near camera modules 205 and 212. Camera modules 205 and 212 may include one or more lenses, image sensors, and / or image signal processors. The flash 213 may include, for example, a light-emitting diode (LED) or a xenon lamp.

[0112] According to an embodiment, the first camera module 205 can be disposed below the display panel of the display 201 using an under-display camera (UDC) scheme. According to an embodiment, two or more lenses (wide-angle and telephoto lenses) and an image sensor can be disposed on one surface of the electronic device 200. According to an embodiment, a plurality of first camera modules 205 can be arranged on a first surface of the electronic device 200 (e.g., the surface of the display screen) using a UDC scheme.

[0113] According to an embodiment, the key input device 217 may be disposed on the side surface 210C of the housing 210. According to an embodiment, the electronic device 200 may not include some or all of the aforementioned key input devices 217, and the excluded key input devices 217 may be implemented on the display 201 in another form, such as soft keys. According to an embodiment, the key input device 217 may be implemented using a pressure sensor included in the display 201.

[0114] According to an embodiment, connectors 208 and 209 may include a first connector hole 208 and / or a second connector hole 209 (or a headphone jack). The first connector hole 208 is capable of accommodating a connector (e.g., a USB connector) for sending power and / or data to / receiving power and / or data from an external electronic device, and the second connector hole 209 is capable of accommodating a connector for sending audio signals to / receiving audio signals from an external electronic device. The first connector hole 208 may include a Universal Serial Bus (USB) Type-A or USB Type-C port. When the first connector hole 208 supports USB Type-C, the electronic device 200 (e.g., ...) Figure 1 The electronic device 101 in the middle can support USB power delivery (PD) charging.

[0115] According to an embodiment, the first camera module 205 in camera modules 205 and 212 and the first sensor module 204 in sensor modules 204 and 219 can be configured to be visible through the display 201.

[0116] According to an embodiment, when the first camera module 205 is arranged using the UDC scheme, the first camera module 205 can be made invisible from the outside.

[0117] According to an embodiment, the first camera module 205 can be arranged to overlap with the display area, and the screen can also be displayed in the display area corresponding to the first camera module 205. The first sensor module 204 can be disposed in the internal space of the electronic device 200 to perform its function, without being visually exposed through the front panel 202.

[0118] Figure 4 This is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0119] Reference Figure 4 Electronic device 400 according to embodiments of the present disclosure (e.g., Figure 1 Electronic device 101 in Figure 2 The electronic device 200 in the middle may include a processor 120 (e.g., Figure 1 The processor 120 and memory 130 (e.g., Figure 1 The memory 130), sensor module 176 and display module 160 (e.g., in the memory 130), sensor module 176 and display module 160) Figure 1 (Display module 160 in the middle).

[0120] According to embodiments, the number of processors 120 may be one or more. For example, processor 120 may have a multi-core processor architecture, such as dual-core, quad-core, hexa-core, or octa-core. Processor 120 can control the operation of electronic devices 101, 200, or 400 by executing instructions stored in memory 130. For example, processor 120 may correspond to multiple processors that jointly perform multiple operations by dividing operations among processors.

[0121] In an embodiment, memory 130 (e.g., Figure 1 The memory 130 may include one or more of the following: high-bandwidth memory (HBM), dynamic random access memory (DRAM), static random access memory (SRAM), phase-change random access memory (PRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), flash memory and / or electrically erasable programmable read-only memory (EEPROM).

[0122] According to one embodiment, the display module 160 (e.g., Figure 1The display module 160 may include a display 410, a display driver IC (hereinafter referred to as "DDIC") 430 for driving the display 410 (e.g., a display driver), a touch circuit 450, a digitizer 460, a digitizer driver 470, and a sensor module 176 (e.g., a touch module 450). Figure 1 (e.g., sensor module 176). For example, all or part of sensor module 176 may be included in display module 160.

[0123] According to an embodiment, DDIC 430 may include an interface module 431 (e.g., an interface circuit), a memory 433 (e.g., a buffer memory), an image processing module 435 (e.g., an image processing circuit), or a mapping module 437 (e.g., a mapping circuit).

[0124] According to an embodiment, DDIC 430 can access an electronic device (e.g., via interface module 431) through interface module 431. Figure 1 Electronic device 101 in Figure 2 and Figure 3 Other components of the electronic device 200 receive image information, including image data or image control signals corresponding to commands for controlling the image data.

[0125] According to an embodiment, it can be obtained from processor 120 (e.g., Figure 1 The processor 120 in the middle (for example, Figure 1 The main processor 121 (e.g., application processor) or an auxiliary processor (e.g., ...) that operates independently of the functions of the main processor 121. Figure 1 The auxiliary processor 123 (e.g., graphics processing unit) in the process receives image information.

[0126] According to an embodiment, the DDIC 430 can communicate with the touch circuit 450 or the sensor module 176 via the interface module 431. Additionally, the DDIC 430 can store at least a portion of the received image information in the memory 433. As an example, the DDIC 430 can store at least a portion of the received image information in the memory 433 on a frame-by-frame basis.

[0127] According to an embodiment, the image processing module 435 may perform preprocessing or postprocessing (e.g., resolution, brightness, or size adjustment) on at least a portion of the image data based at least on the characteristics of the image data or the characteristics of the display 410.

[0128] According to an embodiment, the mapping module 437 can generate voltage or current values ​​corresponding to image data preprocessed or post-processed by the image processing module 435. According to an embodiment, the generation of voltage or current values ​​can be performed at least in part based on the attributes of the pixels of the display 410 (e.g., the arrangement of pixels (RGB stripes or pentile structure) or the size of each sub-pixel).

[0129] According to an embodiment, for example, at least some pixels of the display 410 are driven at least in part based on voltage or current values, thereby enabling the display 410 to display visual information (e.g., text, images, or icons) corresponding to image data.

[0130] According to an embodiment, the touch circuit 450 may include a touch sensor 451 and a touch sensor IC (integrated circuit) 453 for controlling the touch sensor 451.

[0131] According to an embodiment, touch sensor IC 453 can control touch sensor 451 to sense touch input or hover input at a specific location on display 410. For example, touch sensor IC 453 can sense touch input or hover input by measuring changes in signals (e.g., voltage, light intensity, resistance, or charge) at a specific location on display 410. Touch sensor IC 453 can send signals to a processor (e.g., ... Figure 1 The processor 120 in the middle provides information about the sensed touch input or hover input (e.g., location, area, pressure, or time).

[0132] According to an embodiment, at least a portion of the touch circuitry 450 (e.g., touch sensor IC 453) may be included as part of the DDIC 430 or the display 410.

[0133] According to an embodiment, at least a portion of the touch circuit 450 (e.g., touch sensor IC 453) may be included as part of another component (e.g., auxiliary processor 123) disposed outside the display module 160.

[0134] According to an embodiment, the display module 160 may further include at least one sensor (e.g., a fingerprint sensor, iris sensor, pressure sensor, or illuminance sensor) or its control circuitry for the sensor module 176. In this case, at least one sensor or its control circuitry may be embedded in a portion of the display module 160 (e.g., display 410 or DDIC 430) or a portion of the touch circuitry 450.

[0135] For example, if the sensor module 176 embedded in the display module 160 includes a biometric sensor (e.g., a fingerprint sensor), the biometric sensor can obtain biometric information (e.g., a fingerprint image) associated with touch input through a portion of the display 410.

[0136] For example, if the sensor module 176 embedded in the display module 160 includes a pressure sensor, the pressure sensor can obtain pressure information associated with touch input through a portion or the entire area of ​​the display 410.

[0137] According to an embodiment, the touch sensor 451 or the sensor module 176 may be disposed between pixels of the pixel layer of the display 410, or disposed above or below the pixel layer.

[0138] According to an embodiment, the display module 160 may include a digitizer 460 for detecting input (e.g., touch input or hover input) from an electronic pen 300 (e.g., a stylus). For example, the digitizer 460 may convert the analog coordinates (e.g., position) of the electronic pen 300 (e.g., a stylus) into digital coordinate data. The digitizer 460 may transmit the digital coordinate data to the processor 120 and / or the DDIC 430.

[0139] According to an embodiment, processor 120 can obtain digital coordinate data input from digitizer 460. Processor 120 can detect input (e.g., touch input or hover input) via electronic pen 300 (e.g., stylus) based on the digital coordinate data. For example, digitizer 460 may include multiple x-axis channels and multiple y-axis channels. Processor 120 can use sensing signals (e.g., EMR signals) received from the x-axis and y-axis channels disposed in digitizer 460 to sense the position of electronic pen 300 (e.g., stylus). For example, in digitizer 460, multiple x-axis channels and multiple y-axis channels may be arranged sequentially, and processor 120 can use sensing signals (e.g., EMR signals) received from three consecutive channels (e.g., three adjacent channels) to sense the position of electronic pen 300 (e.g., stylus).

[0140] According to an embodiment, the digitizer 460 may be externally invisible due to the display 410, electronic components, and mechanical structure.

[0141] For example, the digitizer 460 can be integrated with or adjacent to the panel display 410. For instance, when the digitizer 460 is applied to the panel display 410, it may include an electromagnetic resonant (EMR) plate (or EMR film). Multiple x-axis channels and multiple y-axis channels for detecting the position of the electronic pen 300 can be disposed on a single EMR plate.

[0142] For example, the digitizer 460 can be integrated with or adjacent to a flexible or foldable display. For example, the digitizer 460 can be positioned along the z-axis (e.g., Figure 2 and Figure 3 The z-axis direction in the display 410 is set (e.g., Figure 2 and Figure 3 Below (e.g., on the lower surface) of the display 201.

[0143] For example, when the digitizer 460 is applied to a flexible or foldable display, the digitizer 460 may include multiple EMR sheets (or EMR films). Multiple x-axis channels and multiple y-axis channels for detecting the position of the electronic pen 300 can be provided on the multiple EMR sheets.

[0144] Figure 5 This is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0145] In the screen-off state (e.g., the screen is off state) of the electronic device 500 according to the embodiment, the user can use an electronic pen (e.g., Figure 2 and Figure 3 The electronic pen 300 generates handwriting. In this disclosure, the content of the handwriting (e.g., information) in the screen-off state can be defined as screen-off handwriting (e.g., screen-off memo).

[0146] refer to Figure 5 Electronic device 500 according to embodiments of the present disclosure (e.g., Figure 1 Electronic device 101 in Figure 2 and Figure 3 Electronic devices 200 Figure 4 The electronic device 400 may include an application 510 (e.g., a note-taking application, note-taking program, memo application, or memo program), a text recognition module 520 (e.g., a character recognition circuit, handwriting recognition circuit, or document recognition circuit), and an artificial intelligence (AI) module 530 (e.g., a generation AI circuit).

[0147] For example, application 510 can be used on a display (e.g., Figure 2 Monitor 201 in Figure 4 The user interface (UI) is displayed on the display 410. The application 510 can display information (e.g., handwriting information, text information, or note screen) through the UI.

[0148] For example, text recognition module 520 can recognize a user's handwriting when the screen is off (e.g., a screen-off memo). Text recognition module 520 can convert the content (e.g., handwriting object, handwriting information, character object, or character information) recognized from the handwriting (e.g., a screen-off memo) in the screen-off state into electronic text or electronic document.

[0149] For example, AI module 530 can learn text (e.g., electronic text or electronic document) obtained based on the user's screen-off handwriting (e.g., screen-off memo) and the user's writing style.

[0150] According to an embodiment, application 510, text recognition module 520, and AI module 530 can be embedded in a processor (e.g., Figure 4 The processor 120 in the middle is used for operation.

[0151] According to an embodiment, at least some of the application 510, text recognition module 520, and AI module 530 may be embedded in processor 120 for operation.

[0152] For example, application 510 and text recognition module 520 can be embedded in processor 120 for operation, while AI module 530 can be located outside processor 120 for operation.

[0153] For example, AI module 530 can be embedded in electronic device 500 (e.g., Figure 4 In the electronic device 400). For example, the AI ​​module 530 can be located in an external electronic device and connected via a network (e.g., Figure 1 The first network 198 or the second network 199 is connected to the electronic device 500. The external electronic device equipped with the AI ​​module 530 can be... Figure 1 At least one of electronic device 102, electronic device 104, or server 108 (e.g., AI server).

[0154] The electronic device 500 according to an embodiment can obtain screen-off handwriting (e.g., screen-off memo) even when the screen is off (e.g., the screen is turned off). For example, the electronic device 500 can provide a text conversion menu (e.g., text conversion function) for converting screen-off handwriting (e.g., screen-off memo) into text (e.g., electronic text or electronic document). A user can request the conversion of screen-off handwriting (e.g., screen-off memo) into text (e.g., electronic text or electronic document) by selecting the text conversion menu (e.g., text conversion function).

[0155] For example, electronic device 500 can obtain stroke information from handwriting with the screen off (e.g., a memo with the screen off). Using the stroke information, electronic device 500 can convert the handwriting with the screen off (e.g., a memo with the screen off) into text (e.g., electronic text or electronic document) and store the text.

[0156] For example, electronic device 500 can correct typos, spacing errors, word order errors, or grammatical errors in text (e.g., electronic text or electronic document).

[0157] For example, when multiple texts (e.g., electronic text or electronic document) are written in a specified format (e.g., email, phone number, or address), electronic device 500 can merge the multiple texts written in the specified format to generate and store a single text (e.g., electronic text or electronic document).

[0158] Figure 6 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0159] Reference Figure 5 and Figure 6 In operation 610, electronic device 500 (e.g., Figure 4 The electronic device 400 can obtain screen-off handwriting (e.g., screen-off memo) even when the screen is off (e.g., the screen is closed). The electronic device 500 can obtain stroke information from the screen-off handwriting (e.g., screen-off memo). The electronic device 500 can use the stroke information to convert the screen-off handwriting (e.g., screen-off memo) into text (e.g., electronic text or electronic document), and can create (e.g., generate) a document containing the text (e.g., electronic document).

[0160] For example, in operation 610, the processor (e.g., Figure 4The processor 120 can obtain screen-off handwriting (e.g., screen-off memo) even when the screen is off (e.g., the screen is closed). The processor 120 can obtain stroke information from the screen-off handwriting (e.g., screen-off memo). The processor 120 can use the stroke information to convert the screen-off handwriting (e.g., screen-off memo) into text (e.g., electronic text or electronic document), and can create (e.g., generate) a document containing the text (e.g., an electronic document).

[0161] For example, in operation 610, the text recognition module 520 can obtain screen-off handwriting (e.g., screen-off memo) even when the screen is off (e.g., the screen is closed). The text recognition module 520 can obtain stroke information from the screen-off handwriting (e.g., screen-off memo). The text recognition module 520 can use the stroke information to convert the screen-off handwriting (e.g., screen-off memo) into text (e.g., electronic text or electronic document), and can create (e.g., generate) a document containing the text (e.g., an electronic document).

[0162] According to an embodiment, in operation 620, the electronic device 500 can store a document including text (e.g., an electronic document) in a database (e.g., ...). Figure 7 Database 720 or Figure 4 The electronic device 500 may store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server). However, this disclosure is not limited to this, and the electronic device 500 may also store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server).

[0163] For example, in operation 620, processor 120 may store documents including text (e.g., electronic documents) in a database (e.g., ...). Figure 7 Database 720 or Figure 4 The processor 120 may store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server). However, this disclosure is not limited thereto, and the processor 120 may also store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server).

[0164] For example, in operation 620, application 510 or text recognition module 520 can store documents containing text (e.g., electronic documents) in a database (e.g., ...). Figure 7 Database 720 or Figure 4 The document is stored in memory 130. However, this disclosure is not limited thereto, and the application 510 or the text recognition module 520 may store documents containing text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server).

[0165] According to an embodiment, in operation 630, the electronic device 500 may begin to learn a document including text (e.g., electronic text or electronic document).

[0166] For example, in operation 630, processor 120 may begin to learn documents that include text (e.g., electronic text or electronic document).

[0167] For example, in operation 630, AI module 530 can begin to learn documents that include text (e.g., electronic text or electronic document).

[0168] According to an embodiment, in operation 640, the electronic device 500 can learn a document (e.g., electronic text or electronic document) including text obtained from a user's screen-off handwriting (e.g., screen-off memo), and learn the user's writing style.

[0169] For example, in operation 640, processor 120 can learn a document (e.g., electronic text or electronic document) including text obtained from a user's screen-off handwriting (e.g., screen-off memo) and learn the user's handwriting style.

[0170] For example, in operation 640, AI module 530 can learn from documents (e.g., electronic text or electronic document) including text obtained from the user's screen-off handwriting (e.g., screen-off memo), and learn the user's writing style.

[0171] According to an embodiment, in operation 650, the electronic device 500 may terminate the learning of documents including text (e.g., electronic text or electronic document) and the user's writing style.

[0172] For example, in operation 650, processor 120 may terminate the learning of documents including text (e.g., electronic text or electronic document) and the user's handwriting style.

[0173] In operation 650, AI module 530 can terminate the learning of documents including text (e.g., electronic text or electronic document) and the user's handwriting style.

[0174] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the electronic device 500 to perform operations. Figure 6 The instructions in operation 610 to 650.

[0175] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the processor 120 may include (e.g., storage) for execution by the processor 120. Figure 6 The instructions in operation 610 to 650.

[0176] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for application 510 to perform. Figure 6 The instructions in operation 610 to 650.

[0177] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the text recognition module 520 to perform. Figure 6 The instructions in operation 610 to 650.

[0178] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory module 530 may include (for example, storage) for the AI ​​module 530 to perform operations. Figure 6 The instructions in operation 610 to 650.

[0179] Figure 7 Figure 700 illustrates the storage of text data (e.g., electronic text documents) in a database. Figure 8 This is a diagram 800 showing an example of learning text and handwriting style in the AI ​​module.

[0180] refer to Figure 5 , Figure 7 and Figure 8 According to an embodiment, electronic devices (e.g., Figure 4 Electronic device 400 or Figure 5 The electronic device 500 can create (e.g., generate) documents (e.g., electronic text or electronic documents) that include text. For example, the electronic device 500 can store documents (e.g., electronic text or electronic documents) that include text in a database 720 (e.g., Figure 4 The electronic device 500 may store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server). However, this disclosure is not limited to this, and the electronic device 500 may also store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server).

[0181] According to an embodiment, electronic device 500 can collect text information used by a user to determine the user's writing style or spoken language style. Electronic device 500 can also collect text information used by a user to determine which language the user primarily uses. Electronic device 500 can display this information on a screen (e.g., Figure 4 The user interface 710 is displayed on the monitor 410 to guide the user through the collection of text information.

[0182] According to an embodiment, electronic device 500 can learn documents including text (e.g., electronic text or electronic document) and the user's handwriting style to personalize and supplement information recognized from screen-off handwriting (e.g., screen-off memos) on a user-by-user basis. For example, electronic device 500 can generate training data for text input.

[0183] According to an embodiment, the processor (e.g., Figure 4 The processor 120 in the process can create (e.g., generate) documents (e.g., electronic text or electronic documents) that include text. For example, the processor (e.g., Figure 4 The processor 120 in the database 720 can store documents containing text (e.g., electronic text or electronic documents) in the database 720. Figure 4 In memory 130). However, this disclosure is not limited thereto, and the processor (e.g., Figure 4 The processor 120 in the middle can store documents including text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server).

[0184] According to an embodiment, the processor (e.g., Figure 4 The processor 120 in the processor can collect text information used by the user to determine the user's writing style or spoken style. Figure 4 The processor 120 in the processor can collect text information used by the user to determine which language the user primarily uses. Figure 4 The processor 120 in the display (e.g., Figure 4 The user interface 710 is displayed on the monitor 410 to guide the user through the collection of text information.

[0185] According to an embodiment, the processor (e.g., Figure 4 The processor 120 in the document can learn from text (e.g., electronic text or electronic document) and the user's handwriting style to personalize and supplement the information recognized from the screen-off handwriting (e.g., screen-off memo) for each user. For example, the processor (e.g., Figure 4 The processor 120 in the middle can generate training data for text input.

[0186] According to an embodiment, application 510 or text recognition module 520 can create (e.g., generate) documents containing text (e.g., electronic text or electronic documents). For example, application 510 or text recognition module 520 can store documents containing text (e.g., electronic text or electronic documents) in database 720 (e.g., ...). Figure 4The document is stored in memory 130. However, this disclosure is not limited thereto, and the application 510 or the text recognition module 520 may store documents containing text (e.g., electronic documents) in an external electronic device or an external server (e.g., an AI server).

[0187] According to an embodiment, the text recognition module 520 or the AI ​​module 530 can collect text information used by the user to determine the user's writing style or spoken language style. The text recognition module 520 or the AI ​​module 530 can collect text information used by the user to determine which language the user primarily uses. Application 510 can be displayed on a screen (e.g., Figure 4 The user interface 710 is displayed on the monitor 410 to guide the user through the collection of text information.

[0188] According to an embodiment, AI module 530 can learn from documents including text (e.g., electronic text or electronic document) and the user's handwriting style to personalize and supplement information recognized from screen-off handwriting (e.g., screen-off memos) for each user. For example, user-inputted text data can be used as input data for AI module 530. AI module 530 can learn the user's handwriting style based on the user-inputted text data. For example, AI module 530 can generate training data for text input.

[0189] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the electronic device 500 to perform operations. Figure 7 and Figure 8 The instructions for operations within.

[0190] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the processor 120 may include (e.g., storage) for execution by the processor 120. Figure 7 and Figure 8 The instructions for operations within.

[0191] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for application 510 to perform. Figure 7 and Figure 8 The instructions for operations within.

[0192] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the text recognition module 520 to perform. Figure 7 and Figure 8 The instructions for operations within.

[0193] According to an embodiment, the memory (e.g., Figure 4The memory 130 in the memory module 530 may include (for example, storage) for the AI ​​module 530 to perform operations. Figure 7 and Figure 8 The instructions for operations within.

[0194] Figures 9 to 12 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0195] Reference Figure 5 and Figures 9 to 12 According to an embodiment, in operation 910, the electronic device 500 can create (e.g., obtain) screen-off handwriting (e.g., screen-off memo) while the screen is off (e.g., the screen is in a state where the screen is off).

[0196] For example, in operation 910, the processor (e.g., Figure 4 The processor 120 in the middle can create (e.g., obtain) screen-off handwriting (e.g., screen-off memo) while the screen is off (e.g., the screen is turned off).

[0197] For example, in operation 910, application 510 or text recognition module 520 can create (e.g., obtain) screen-off handwriting (e.g., screen-off memo) in a screen-off state (e.g., the state where the screen is off).

[0198] Figure 13 This is a diagram showing the storage of screen-off handwriting (e.g., screen-off memos) and stroke information obtained from screen-off handwriting in a database.

[0199] Reference Figure 5 , Figures 9 to 12 and Figure 13 According to an embodiment, in operation 920, the electronic device 500 can obtain stroke information from screen-off handwriting (e.g., screen-off memos). Subsequently, the electronic device 500 can store the screen-off handwriting 1310 (e.g., screen-off memos) in a database 720 (e.g., ...). Figure 4 The electronic device 500 may store the screen-off handwriting 1310 (e.g., a screen-off memo) in an external electronic device or an external server (e.g., an AI server). However, this disclosure is not limited to this, and the electronic device 500 may also store the screen-off handwriting 1310 (e.g., a screen-off memo) in an external electronic device or an external server.

[0200] For example, in operation 920, processor 120 can store screen-off handwriting 1310 (e.g., screen-off memo) in database 720 (e.g., ...). Figure 4The processor 120 may store the screen-off handwriting 1310 (e.g., a screen-off memo) in an external electronic device or an external server (e.g., an AI server). The electronic device 500 may provide the stroke information obtained from the screen-off handwriting (e.g., a screen-off memo) to the text recognition module 520.

[0201] For example, in operation 920, processor 120 can obtain stroke information from screen-off handwriting (e.g., screen-off memo). Subsequently, application 510 or text recognition module 520 can store screen-off handwriting 1310 (e.g., screen-off memo) in database 720 (e.g., ...). Figure 4 The text is stored in memory 130. However, this disclosure is not limited thereto, and the application 510 or the text recognition module 520 may store the screen-off handwriting 1310 (e.g., a screen-off memo) in an external electronic device or an external server (e.g., an AI server). The processor 120 may provide the stroke information obtained from the screen-off handwriting (e.g., a screen-off memo) to the text recognition module 520.

[0202] After performing operation 920, you can perform operation 1010.

[0203] Figure 14 This is a diagram showing the identification of words in handwriting when the screen is off (e.g., a memo when the screen is off).

[0204] Reference Figure 5 , Figures 9 to 12 and Figure 14 According to an embodiment, in operation 1010, the electronic device 500 can use stroke information to identify word 1410 from screen-off handwriting (e.g., screen-off memo).

[0205] For example, in operation 1010, processor 120 can use stroke information to identify word 1410 from screen-off handwriting (e.g., screen-off memo).

[0206] For example, in operation 1010, the text recognition module 520 can use stroke information to recognize word 1410 from screen-off handwriting (e.g., screen-off memo).

[0207] Figure 15 This is a diagram showing the lines in a screen-off handwriting (e.g., a screen-off memo).

[0208] Reference Figure 5 , Figures 9 to 12 and Figure 15According to an embodiment, in operation 1020, the electronic device 500 can recognize lines L1 and L2 containing words written in screen-off handwriting (e.g., screen-off memos). The electronic device 500 can distinguish between words written in the first line L1 and words written in the second line L2.

[0209] For example, in operation 1020, processor 120 can identify lines L1 and L2 containing words written in screen-off handwriting (e.g., screen-off memos). Processor 120 can distinguish between words written in the first line L1 and words written in the second line L2.

[0210] For example, in operation 1020, the text recognition module 520 can recognize lines L1 and L2 containing words written in screen-off handwriting (e.g., screen-off memos). The text recognition module 520 can distinguish between words written in the first line L1 and words written in the second line L2.

[0211] Figure 16 This is a diagram showing the line recognition results of handwriting with the screen off (e.g., a memo with the screen off).

[0212] Combination Figure 15 and Figure 16 As described in the embodiment, since the word "the" 1510 written in the first line L1 and the word "testing" 1520 written in the second line L2 at least partially overlap each other, the electronic device 500 can recognize that the words "the" 1510 and "testing" 1520 are written in the same line. Similarly, since the word "memo" 1530 written in the first line L1 and the word "off" 1540 written in the second line L2 at least partially overlap each other, the electronic device 500 can recognize that the words "memo" 1530 and "off" 1540 are written in the same line. By reflecting such line recognition results, the electronic device 500 can recognize that the words 1620 of the screen-off handwriting (e.g., screen-off memo) initially written across two lines are intended to be in the same line, and can arrange the words in a single line to display them through the user interface, as shown in 1610.

[0213] According to an embodiment, since the word "the" 1510 written in the first line L1 and the word "testing" 1520 written in the second line L2 at least partially overlap each other, the processor 120 can recognize that the words "the" 1510 and "testing" 1520 are written in the same line. Similarly, since the word "memo" 1530 written in the first line L1 and the word "off" 1540 written in the second line L2 at least partially overlap each other, the processor 120 can recognize that the words "memo" 1530 and "off" 1540 are written in the same line. By reflecting such line recognition results, the processor 120 can recognize that the words 1620 of the screen-off handwriting (e.g., screen-off memo) initially written across two lines are intended to be in the same line, and can arrange the words in a single line to display them through the user interface, as shown in 1610.

[0214] According to an embodiment, since the word "the" 1510 written in the first line L1 and the word "testing" 1520 written in the second line L2 at least partially overlap each other, the text recognition module 520 can recognize that the words "the" 1510 and "testing" 1520 are written in the same line. Similarly, since the word "memo" 1530 written in the first line L1 and the word "off" 1540 written in the second line L2 at least partially overlap each other, the text recognition module 520 can recognize that the words "memo" 1530 and "off" 1540 are written in the same line. By reflecting such line recognition results, the text recognition module 520 can recognize that the words 1620 of the screen-off handwriting (e.g., screen-off memo) initially written across two lines are intended to be in the same line, and can arrange the words in a single line to display them through the user interface, as shown in 1610.

[0215] According to an embodiment, the electronic device 500 can receive user input regarding text 1610 changed from the handwriting memo 1620. For example, when the user input is received in a user interface displaying the text, the electronic device 500 can change the user interface to display the handwriting memo 1620 corresponding to the text.

[0216] According to an embodiment, processor 120 can receive user input regarding text 1610 changed from handwriting memo 1620. For example, when the user input is received in a user interface displaying the text, processor 120 can change the user interface to display the handwriting memo 1620 corresponding to the text.

[0217] Figure 17 This is a diagram illustrating the post-processing results of an electronic device according to an embodiment of the present disclosure.

[0218] Reference Figure 5 , Figures 9 to 12 and Figure 17 According to an embodiment, in operation 1030, electronic device 500 can perform post-processing to analyze the words in the screen-off handwriting (e.g., screen-off memo) and generate (e.g., obtain) information in a specified format. After line recognition, electronic device 500 can determine (e.g., confirm) whether it can be combined with another line to generate meaningful information in a specified format (e.g., phone number, link, address, email). Electronic device 500 can obtain phone number 1712 from the screen-off handwriting (e.g., screen-off memo). In this case, if the phone number 1712 in the screen-off handwriting (e.g., screen-off memo) is written on two lines, electronic device 500 can determine whether it corresponds to information in a specified format by concatenating the corresponding line and the next line. If it corresponds to information in a specified format by concatenating the corresponding line and the next line, electronic device 500 can convert it into a single-line electronic text phone number 1722. Electronic device 500 can assign application action attributes to the text phone number 1722, such that when the text phone number 1722 is selected, a telephone application is executed to make a call. Electronic device 500 can store phone numbers 1712 (screen off handwriting, e.g., screen off memos) and phone numbers 1722 (single-line electronic text) in a database (e.g., Figure 13 In the database 720).

[0219] According to an embodiment, in operation 1030, the processor (e.g., Figure 4The processor 120 can perform post-processing to analyze the words in the screen-off handwriting (e.g., a screen-off memo) and generate (e.g., obtain) information in a specified format. After line recognition, the processor 120 can determine (e.g., confirm) whether it can be combined with another line to generate meaningful information in a specified format (e.g., a phone number, link, address, email). The processor 120 can obtain a phone number 1712 from the screen-off handwriting (e.g., a screen-off memo). In this case, if the phone number 1712 in the screen-off handwriting (e.g., a screen-off memo) is written on two lines, the processor 120 can determine whether it corresponds to information in a specified format by concatenating the corresponding line and the next line. If it corresponds to information in a specified format by concatenating the corresponding line and the next line, the processor 120 can convert it into a single-line electronic text phone number 1722. The processor 120 can assign application action attributes to the text phone number 1722, such that when the text phone number 1722 is selected, a phone application is executed to make a call. Processor 120 can store phone numbers 1712 (screen off handwriting, e.g., screen off memos) and phone numbers 1722 (single-line electronic text) in a database (e.g., Figure 13 In the database 720).

[0220] According to the implementation method, in operation 1030, the text recognition module (e.g., Figure 5 The text recognition module 520 can perform post-processing to analyze the words in the screen-off handwriting (e.g., a screen-off memo) and generate (e.g., obtain) information in a specified format. After line recognition, the text recognition module 520 can determine (e.g., confirm) whether it can generate meaningful information in a specified format (e.g., a phone number, link, address, email) by combining it with another line. The text recognition module 520 can obtain a phone number 1712 from the screen-off handwriting (e.g., a screen-off memo). In this case, if the phone number 1712 in the screen-off handwriting (e.g., a screen-off memo) is written on two lines, the text recognition module 520 can determine whether it corresponds to information in a specified format by concatenating the corresponding line and the next line. If it corresponds to information in a specified format by concatenating the corresponding line and the next line, the text recognition module 520 can convert it into a single-line electronic text phone number 1722. The text recognition module 520 can assign application action attributes to the phone number 1722 in the text, such that when the phone number 1722 in the text is selected, a telephone application is executed to make a call. The text recognition module 520 can store the phone number 1712 in handwriting (e.g., a screen-off memo) and the phone number 1722 in single-line electronic text in a database (e.g., Figure 13 In the database 720).

[0221] After performing operation 1030, operation 930 can be performed.

[0222] According to an embodiment, in operation 930, the electronic device 500 can obtain stroke information from screen-off handwriting (e.g., screen-off memos). Using the stroke information, the electronic device 500 can recognize the screen-off handwriting (e.g., screen-off memos) as text (e.g., electronic text or electronic document). In this case, the electronic device 500 can obtain text and text information recognized on a line-by-line basis. The electronic device 500 can create (e.g., generate) a document including text (e.g., an electronic document).

[0223] According to an embodiment, in operation 930, processor 120 can obtain stroke information from screen-off handwriting (e.g., screen-off memos). Using the stroke information, processor 120 can recognize screen-off handwriting (e.g., screen-off memos) as text (e.g., electronic text or electronic document). In this case, processor 120 can obtain text and text information recognized on a line-by-line basis. Processor 120 can write (e.g., generate) a document (e.g., an electronic document) including text.

[0224] According to an embodiment, in operation 930, the text recognition module 520 can obtain stroke information from screen-off handwriting (e.g., screen-off memos). Using the stroke information, the text recognition module 520 can recognize the screen-off handwriting (e.g., screen-off memos) as text (e.g., electronic text or electronic document). In this case, the text recognition module 520 can obtain information about the text recognized line by line. The text recognition module 520 can write (e.g., generate) a document containing text (e.g., an electronic document).

[0225] According to an embodiment, in operation 940, the electronic device 500 can store the information of the recognized text in a database (e.g., Figure 7 In the database 720).

[0226] According to an embodiment, in operation 940, the processor 120 may store the information of the recognized text in a database (e.g., Figure 7 In the database 720).

[0227] According to an embodiment, in operation 940, the text recognition module 520 can store the information of the recognized text in a database (e.g., Figure 7 In the database 720).

[0228] Figure 18 This is a diagram illustrating an example of using prompts for document organization on an electronic device according to an embodiment of the present disclosure. Figure 19This is a diagram illustrating the document organization results of an electronic device according to an embodiment of the present disclosure. Figure 20 This is a diagram illustrating the document organization results of an electronic device according to an embodiment of the present disclosure.

[0229] Combination Figures 18 to 20 As described in the embodiment, in operation 950, electronic device 500 can check for errors (e.g., typos, spacing, word order, or grammatical errors) in text identified through post-processing, and organize the document by correcting the text errors.

[0230] For example, electronic device 500 can send data such as... Figure 18 The prompt shown in 1800 is to check for errors in the text (such as typos, spacing, word order, or grammatical errors) and to organize the document by correcting text errors.

[0231] For example, if the text contains grammatical errors or includes words that are not appropriate for the context, the electronic device 500 can organize the document by correcting the text errors.

[0232] For example, if the words included in the text are different from the writing style previously used by the user, the electronic device 500 can organize the document by correcting the words included in the text to match the writing style previously used by the user.

[0233] In operation 950, processor 120 can check for errors in the text identified through post-processing (e.g., typos, spacing, word order, or grammatical errors) and organize the document by correcting text errors.

[0234] For example, processor 120 can... Figure 18 The prompt 1800 shown is sent to the AI ​​module 530 to check for errors in the text (e.g., typos, spacing, word order, or grammatical errors) and to organize the document by correcting text errors.

[0235] For example, if the text contains words with grammatical errors or includes words that do not fit the context, the processor 120 can organize the document by correcting the text errors.

[0236] For example, if the words included in the text are different from the writing style previously used by the user, the processor 120 can organize the document by correcting the words included in the text to match the writing style previously used by the user.

[0237] After executing operation 950, operation 1110 can be executed.

[0238] like Figure 19As shown, in operation 1110, electronic device 500 can supplement and organize text. For example, if the word 1910 contained in text 2010 is written on two lines, electronic device 500 can correct them into a single line and generate corrected text 1920. Electronic device 500 can be displayed on a screen (e.g., Figure 4 The corrected text 1920 is displayed on the monitor 410.

[0239] like Figure 20 As shown, in operation 1110, if the word "He llo I'm testing theescreen off memo" contained in text 2010 contains typos and / or spacing errors, the electronic device 200 can correct it to " Hello I'm testing the The screen off memo is then displayed, and the corrected text 2020 is generated (e.g., electronic text or electronic document). The electronic device 500 can display the corrected text 2020 on the display 410.

[0240] like Figure 19 As shown, in operation 1110, if the word 1910 contained in the text 2010 is written on two lines, the processor 120 can correct them into a single line and generate the corrected text 1920. The processor 120 can be accessed via a display (e.g., Figure 4 The corrected text 1920 is displayed on the monitor 410.

[0241] like Figure 20 As shown, in operation 1110, if the text 2010 contains the word " He llo I'm testing thee If there are typos and spacing errors in the "screen off memo" text, the processor 120 can correct them to " Hello I'm testing the The processor 120 can display the corrected text 2020 (e.g., electronic text or electronic document) by clicking "screen off memo".

[0242] After operations 950 and 1110, operation 960 can be performed.

[0243] According to an embodiment, in operation 960, electronic device 500 can determine whether the text corrected by AI module 530 is more than one (e.g., two or more).

[0244] As a result of operation 960, if the text corrected by AI module 530 is more than one (e.g., two or more), then electronic device 500 can perform operation 970.

[0245] According to an embodiment, in operation 970, the electronic device 500 can merge two or more segments of text into one segment of text.

[0246] Figure 21 This is a diagram illustrating an example of a document merging prompt using an electronic device according to an embodiment of the present disclosure. Figure 22 This is a diagram illustrating the document merging result of an electronic device according to an embodiment of the present disclosure.

[0247] Combination Figure 21 and Figure 22 The description states that if two or more segments of text 2210 are present and corrected by the AI ​​module 530, the electronic device 500 can send a message to the AI ​​module 530 such as... Figure 21 The prompt shown is 2100.

[0248] According to an embodiment, in operation 960, processor 120 may determine whether the text corrected by AI module 530 is more than one (e.g., two or more).

[0249] As a result of operation 960, if the text corrected by AI module 530 is more than one (e.g., two or more), then processor 120 may perform operation 970.

[0250] According to an embodiment, in operation 970, processor 120 can merge two or more segments of text into a single segment of text.

[0251] Combination Figure 21 and Figure 22 The description states that if there are two or more segments of text 2210 corrected by the AI ​​module 530, the processor 120 can send a message to the AI ​​module 530 such as... Figure 21 The prompt shown is 2100.

[0252] According to an embodiment, in operation 1210, AI module 530 may perform an operation based on prompt 2100 and determine whether two or more text segments 2210 include a specified format (e.g., email, phone number, address). If two or more text segments 2210 include the specified format (e.g., email, phone number, address), then AI module 530 may merge the two or more text segments 2210 into a single text segment 2220.

[0253] For example, AI module 530 can perform an operation based on prompt 2100 to analyze the similarity of two or more text segments 2210, and if the two or more text segments 2210 are similar, merge them into a single text segment 2220.

[0254] For example, if two or more text segments 2210 contain telephone numbers in a specified format, the AI ​​module 530 can merge the two or more telephone number texts into a single telephone number text.

[0255] For example, if two or more text segments 2210 contain emails in a specified format, the AI ​​module 530 can merge the two or more email texts into a single email text.

[0256] For example, if two or more text segments 2210 contain addresses in a specified format, the AI ​​module 530 can merge the two or more address texts into a single address text.

[0257] According to an embodiment, in operation 980, the electronic device 500 can display the merged single-segment text 2220 via the display 410.

[0258] According to an embodiment, in operation 980, processor 120 can display merged single-segment text 2220 via display 410.

[0259] According to an embodiment, in operation 980, application 510 can display the merged single-segment text 2220 via display 410.

[0260] According to an embodiment, the electronic device 500 can apply colors matching previously identified words to the organized and supplementary content of the text, allowing a user to verify the organized and supplementary content. The electronic device 500 can also assign link information to the organized and supplementary content of the text, enabling a user to verify the organized and supplementary content.

[0261] According to an embodiment, processor 120 may apply colors matching previously identified words to the organized and supplementary content of the text, allowing a user to verify the organized and supplementary content of the text. Processor 120 may also assign link information to the organized and supplementary content of the text, allowing a user to verify the organized and supplementary content of the text.

[0262] According to an embodiment, application 510 can apply colors matching previously identified words to the organized and supplementary content of the text, allowing the user to verify the organized and supplementary content of the text. Application 510 can assign link information to the organized and supplementary content of the text, allowing the user to verify the organized and supplementary content of the text.

[0263] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the electronic device 500 to perform operations. Figures 9 to 12 The instructions for operations within.

[0264] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the electronic device 500 to perform operations. Figures 13 to 22 The instructions for operations within.

[0265] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the processor 120 may include (e.g., storage) for execution by the processor 120. Figures 9 to 12 The instructions for operations within.

[0266] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the processor 120 may include (e.g., storage) for execution by the processor 120. Figures 13 to 22 The instructions for operations within.

[0267] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for application 510 to perform. Figures 9 to 12 The instructions for operations within.

[0268] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for application 510 to perform. Figures 13 to 22 The instructions for operations within.

[0269] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the text recognition module 520 to perform. Figures 9 to 12 The instructions for operations within.

[0270] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the text recognition module 520 to perform. Figures 13 to 22 The instructions for operations within.

[0271] According to an embodiment, the memory (e.g., Figure 4The memory 130 in the memory module 530 may include (for example, storage) for the AI ​​module 530 to perform operations. Figures 9 to 12 The instructions for operations within.

[0272] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory module 530 may include (for example, storage) for the AI ​​module 530 to perform operations. Figures 13 to 22 The instructions for operations within.

[0273] For example, it can be omitted Figures 9 to 12 At least some of the operations in the process. For example, they can be executed sequentially. Figures 9 to 12 Operations within the context. For example, they can be executed simultaneously (e.g., in parallel). Figures 9 to 12 At least some of the operations in.

[0274] For example, it can be omitted Figures 13 to 22 At least some of the operations in the process. For example, they can be executed sequentially. Figures 13 to 22 Operations within the context. For example, they can be executed simultaneously (e.g., in parallel). Figures 13 to 22 At least some of the operations in.

[0275] Figure 23 This is a diagram illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0276] refer to Figure 4 and Figure 23 According to an embodiment, in operation 2310, the electronic device 400 can obtain the user's handwriting (e.g., obtain the handwriting written on the display) through user input (e.g., electronic pen input) when the screen of the display 410 is off.

[0277] According to an embodiment, in operation 2310, the processor 120 can obtain the user's handwriting (e.g., obtain the handwriting written on the display) through user input (e.g., electronic pen input) when the display 410 is in a screen-off state.

[0278] According to an embodiment, in operation 2320, the electronic device 400 can obtain a first handwriting memo and a second handwriting memo related to the handwriting obtained when the display 410 is in a screen-off state. The electronic device 400 can convert the first handwriting memo and the second handwriting memo into electronic text.

[0279] According to an embodiment, in operation 2320, processor 120 can obtain a first handwriting memo and a second handwriting memo related to the handwriting obtained while the display 410 is in a screen-off state. Electronic device 400 can convert the first handwriting memo and the second handwriting memo into electronic text.

[0280] According to an embodiment, in operation 2330, the electronic device 400 can store electronic text in the memory 130 (e.g., a database, a server).

[0281] According to an embodiment, in operation 2330, processor 120 may store electronic text in memory 130 (e.g., database, server).

[0282] According to an embodiment, in operation 2340, the electronic device 400 can display electronic text on the display 410. Simultaneously, the electronic device 400 can display a first handwriting memo and a second handwriting memo on the display 410.

[0283] According to an embodiment, in operation 2340, the processor 120 can display electronic text on the display 410. Simultaneously, the processor 120 can display a first handwriting memo and a second handwriting memo on the display 410.

[0284] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the memory may include (for example, storage) for the electronic device 400 to perform operations. Figure 23 The instructions for operations within.

[0285] According to an embodiment, the memory (e.g., Figure 4 The memory 130 in the processor 120 may include (e.g., storage) for execution by the processor 120. Figure 23 The instructions for operations within.

[0286] For example, it can be omitted Figure 23 At least some of the operations in the process. For example, they can be executed sequentially. Figure 23 The operations within. For example. Figure 23 At least some of the operations in the process can be executed simultaneously (e.g., in parallel).

[0287] Electronic devices according to embodiments of the present disclosure (e.g., Figure 2 and Figure 3 Electronic devices 200 Figure 4 Electronic devices 400 in Figure 5 The electronic device 500 in the middle may include a display module (e.g., Figure 4 The display module 160 in the middle), the processor (e.g., Figure 4 The processor 120 in the memory (e.g., Figure 4 The memory 130 in the memory module 160 can be configured to accept user input (e.g., when the screen is off). Figure 2 and Figure 3The electronic device 200, 400, or 500 can acquire the user's handwriting using an electronic pen 300. The processor 120 can be configured to control the operation of the display module 160. The memory 130 can include a first handwriting memo and a second handwriting memo related to the handwriting acquired in a screen-off state, and can be operatively connected to the processor 120 and include instructions. When the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to generate electronic text from the first and second handwriting memos related to the handwriting acquired in a screen-off state. When the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to store the electronic text in the memory 130. When the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to simultaneously display the electronic text on the display module 160 along with the first and second handwriting memos.

[0288] According to an embodiment, when instructions are executed by processor 120, electronic devices 200, 400, or 500 can be configured to group objects of handwriting existing on the same line. When instructions are executed by processor 120, electronic devices 200, 400, or 500 can be configured to generate electronic text for each line of handwriting objects.

[0289] According to an embodiment, when the processor 120 executes instructions, the electronic device 200, 400, or 500 can be configured to learn the user's handwriting and writing style. When the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to generate electronic text by reflecting the results of the learning.

[0290] According to an embodiment, when instructions are executed by processor 120, electronic device 200, 400, or 500 can be configured to generate electronic text by correcting grammatical errors and sentence context errors in handwriting.

[0291] According to an embodiment, when the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to determine the similarity between a first handwriting memo and a second handwriting memo. When the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to merge the first handwriting memo and the second handwriting memo into a single electronic text if the first handwriting memo and the second handwriting memo meet a predetermined similarity criterion.

[0292] According to an embodiment, when the instructions are executed by the processor 120, the electronic device 200, 400 or 500 can be configured to generate a single electronic text from the first handwriting memo and the second handwriting memo written in multiple lines and including a predetermined format.

[0293] According to an embodiment, when the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to assign application action attributes to the electronic text, making the application associated with the electronic text executable.

[0294] According to an embodiment, when the instructions are executed by the processor 120, the electronic device 200, 400, or 500 can be configured to display electronic text over the first handwriting memo.

[0295] According to embodiments of this disclosure, the operation method of electronic device 200, 400, or 500 can obtain user handwriting input on display 410 in a screen-off state. The operation method can convert a first handwriting memo and a second handwriting memo related to the handwriting obtained in the screen-off state into electronic text. The operation method can store the electronic text in memory 130. The operation method can simultaneously display the electronic text on display 410 along with the first and second handwriting memos.

[0296] According to an embodiment, the method can group handwriting objects existing on the same line by analyzing the handwriting. The method can then generate electronic text for each line of handwriting objects.

[0297] According to an embodiment, the operation method can learn a user's handwriting and writing style. The operation method can then generate electronic text by reflecting the results of this learning.

[0298] According to an embodiment, the method can generate electronic text by correcting grammatical errors and sentence context errors in handwriting.

[0299] According to an embodiment, the operation method can determine the similarity between a first handwriting memo and a second handwriting memo. If the first handwriting memo and the second handwriting memo meet a predetermined similarity criterion, the operation method can merge the first handwriting memo and the second handwriting memo into a single electronic text.

[0300] According to an embodiment, when the first handwriting memo and the second handwriting memo are written in multiple lines and include a predetermined format, the operation method can generate the first handwriting memo and the second handwriting memo written in multiple lines into a single electronic text.

[0301] According to an embodiment, the operation method can assign application action attributes to electronic text, making the application associated with the electronic text executable.

[0302] According to an embodiment, the operation method can display electronic text above the first handwriting memo.

[0303] In the recording medium storing instructions readable by the processor 120 of electronic devices 200, 400, or 500, when the processor 120 executes the instructions, the instructions can cause electronic devices 200, 400, or 500 to obtain user handwriting input on display 410 while the screen is off. When executed by the processor 120, the instructions can cause electronic devices 200, 400, or 500 to convert a first handwriting memo and a second handwriting memo related to the handwriting obtained while the screen is off into electronic text. When executed by the processor 120, the instructions can cause electronic devices 200, 400, or 500 to store the electronic text in memory 130. When executed by the processor 120, the instructions can cause electronic devices 200, 400, or 500 to simultaneously display the electronic text and the first and second handwriting memos on display 410.

[0304] According to an embodiment, the instructions, when executed by the processor 120, can cause the electronic devices 200, 400, or 500 to group handwriting objects existing on the same line by analyzing the handwriting. When executed by the processor 120, the instructions can cause the electronic devices 200, 400, or 500 to generate electronic text for each line of handwriting objects.

[0305] According to an embodiment, when executed by processor 120, the instructions can cause electronic devices 200, 400, or 500 to learn a user's handwriting and writing style. When executed by processor 120, the instructions can cause electronic devices 200, 400, or 500 to generate electronic text by reflecting the learning results.

[0306] According to an embodiment, when the first handwriting memo and the second handwriting memo are written in multiple lines and include a predetermined format, the instructions, when executed by the processor 120, can cause the electronic device 200, 400, or 500 to generate the first handwriting memo and the second handwriting memo written in multiple lines into a single electronic text.

[0307] An electronic device and its operating method according to embodiments of the present disclosure can convert a handwritten memo with the screen off into an electronic text document and store the electronic text document in a memory (e.g., a database or server).

[0308] The electronic device and its operating method according to embodiments of the present disclosure can convert a handwritten memo with the screen off into an electronic text document and display the electronic text document on a display.

[0309] The electronic device and its operating method according to embodiments of the present disclosure can convert a screen-off memo into multiple electronic text documents and merge multiple electronic text documents.

[0310] The electronic device and its operating method according to embodiments of the present disclosure can convert a screen-off memo into multiple electronic text documents, merge the multiple electronic text documents, and display them on a display.

[0311] The effects that can be obtained from this disclosure are not limited to those described above, and those skilled in the art to which this disclosure pertains will clearly understand from the above description other effects not mentioned herein.

Claims

1. An electronic device (200, 400, 500), comprising: The display module (160) is configured to obtain the user's handwriting through user input when the screen is off; The processor (120) is configured to control the operation of the display module (160); and The memory (130), operably connected to the processor (120) and including instructions, The memory (130) includes a first handwriting memo and a second handwriting memo related to the handwriting obtained in the screen-off state of the display module, and When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured as follows: The first handwriting memo and the second handwriting memo, which are related to the handwriting obtained in the screen-off state, are generated as electronic text. The electronic text is stored in the memory (130), and The electronic text is displayed simultaneously on the display module (160) along with the first handwriting memo and the second handwriting memo.

2. The electronic device (200, 400, 500) according to claim 1, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: Grouping objects of the handwriting that exist on the same line, and For each line of handwriting, generate an electronic text object.

3. The electronic device (200, 400, 500) according to claim 1 or 2, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: Learn users' handwriting and writing style, and The electronic text is generated by reflecting the results of the learning process.

4. The electronic device (200, 400, 500) according to claim 3, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: The electronic text is generated by correcting grammatical errors and sentence context errors in the handwriting.

5. The electronic device (200, 400, 500) according to any one of claims 1 to 4, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: Determine the similarity between the first handwriting memo and the second handwriting memo, and If the first handwriting memo and the second handwriting memo meet a predetermined similarity criterion, then the first handwriting memo and the second handwriting memo are merged into a single electronic text.

6. The electronic device (200, 400, 500) according to any one of claims 1 to 5, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: When the first handwriting memo and the second handwriting memo are written on multiple lines and include a predetermined format, The first handwriting memo and the second handwriting memo, written in multiple lines, are generated into a single electronic text.

7. The electronic device (200, 400, 500) according to any one of claims 1 to 6, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: Assign application action attributes to the electronic text so that the application associated with the electronic text is executable.

8. The electronic device (200, 400, 500) according to claim 1, wherein, When the instructions are executed by the processor (120), the electronic devices (200, 400, 500) are configured to: The electronic text is displayed above the first handwriting memo.

9. A method of operating an electronic device (200, 400, 500), comprising: Obtain user handwriting on the display (410) by user input while the screen is off; The first and second handwriting memos related to the handwriting obtained in the screen-off state are converted into electronic text; The electronic text is stored in the memory (130); and The electronic text (410) is displayed on the display simultaneously with the first handwriting memo and the second handwriting memo.

10. A method of operating the electronic device (200, 400, 500) according to claim 9, comprising: By analyzing the handwriting, objects with handwriting that exist on the same line are grouped together; and For each line of handwriting, generate an electronic text object.

11. A method of operating the electronic device (200, 400, 500) according to claim 9 or 10, comprising: Learn users' handwriting and writing style, and The electronic text is generated by reflecting the results of the learning process.

12. The method of operating the electronic device (200, 400, 500) according to claim 11, comprising: The electronic text is generated by correcting grammatical errors and sentence context errors in the handwriting.

13. A method of operating the electronic device (200, 400, 500) according to any one of claims 9 to 12, comprising: Determine the similarity between the first handwriting memo and the second handwriting memo, and If the first handwriting memo and the second handwriting memo meet a predetermined similarity criterion, then the first handwriting memo and the second handwriting memo are merged into a single electronic text.

14. A method of operating the electronic device (200, 400, 500) according to any one of claims 9 to 13, comprising: When the first handwriting memo and the second handwriting memo are written on multiple lines and include a predetermined format, The first handwriting memo and the second handwriting memo, written in multiple lines, are generated into a single electronic text.

15. A method of operating the electronic device (200, 400, 500) according to any one of claims 9 to 14, comprising: Assign application action attributes to the electronic text so that the application associated with the electronic text is executable.

Images