Electronic device for transforming handwriting input, operating method thereof, and recording medium therefor

Abstract

An electronic device according to an embodiment comprises: a touch screen; a memory storing instructions; and at least one processor, wherein the instructions, when executed by the at least one processor, cause the electronic device to: obtain a first handwriting input including a plurality of strokes through the touch screen; obtain coordinate information of the plurality of strokes; obtain feature information related to the plurality of strokes, wherein the feature information includes information about an angle at which the first handwriting input is inclined with respect to a reference line; input the coordinate information and the feature information to a generative AI model to obtain a plurality of handwritings in which a plurality of pieces of text automatically completed from at least one piece of text corresponding to the first handwriting input are transformed on the basis of the feature information; and display the first handwriting input and the plurality of handwritings on the touch screen.

Description

Electronic device for transforming handwriting input, method of operation thereof and recording medium

[0001] Embodiments of the present disclosure relate to an electronic device for transforming handwriting input, a method of operating the same, and a recording medium.

[0002] User equipment such as smartphones, tablets, and wearable devices may provide various functions such as music playback, navigation, short-range wireless communication (e.g., Bluetooth, Wi-Fi, or near field communication (NFC)), fingerprint recognition, photo (still image) or video recording, or electronic payment functions.

[0003] An electronic device can detect handwriting input using an input device having a pen function (referred to herein as a "stylus pen" for convenience of explanation). The input method using a stylus pen allows for more precise touch input than the touch input method using a finger, and thus can be more usefully applied when performing applications such as note-taking or sketching.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] According to one embodiment, an electronic device may include a touch screen, a memory storing instructions, and at least one processor. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain a first handwriting input including a plurality of strokes through the touch screen. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain coordinate information of the plurality of strokes. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain feature information related to the plurality of strokes, wherein the feature information may include information indicating an angle at which the first handwriting input is inclined with respect to a reference line. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to provide the coordinate information and the feature information to a generative AI model, thereby obtaining a plurality of handwritings in which a plurality of texts are automatically completed from at least one text corresponding to the first handwriting input and are transformed based on the feature information. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to display the first handwriting input and the plurality of handwritings on the touch screen.

[0006] According to one embodiment, a method of operating an electronic device may include an operation of obtaining a first handwriting input including a plurality of strokes through a touch screen included in the electronic device. According to one embodiment, the method of operating the electronic device may include an operation of obtaining coordinate information of the plurality of strokes. According to one embodiment, the method of operating the electronic device may include an operation of obtaining feature information related to the plurality of strokes, wherein the feature information may include information indicating an angle at which the first handwriting input is inclined with respect to a reference line. According to one embodiment, the method of operating the electronic device may include an operation of providing the coordinate information and the feature information to a generative AI model (340) to obtain a plurality of handwritings in which a plurality of texts are automatically completed from at least one text corresponding to the first handwriting input and are transformed based on the feature information. According to one embodiment, the method of operating the electronic device may include an operation of displaying the first handwriting input and the plurality of handwritings on the touch screen.

[0007] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of obtaining a first handwriting input including a plurality of strokes through a touch screen included in the electronic device. According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of the electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of obtaining coordinate information of the plurality of strokes. According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of obtaining feature information related to the plurality of strokes, wherein the feature information may include information indicating an angle at which the first handwriting input is inclined with respect to a reference line. According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of providing the coordinate information and the feature information to a generative AI model, thereby obtaining a plurality of handwritings in which a plurality of texts are automatically completed from at least one text corresponding to the first handwriting input and are transformed based on the feature information.According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of displaying the first handwriting input and the plurality of handwritings on the touch screen.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0009] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0010] FIG. 3 is a schematic block diagram of a memory according to one embodiment.

[0011] FIG. 4 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to execute an automatic completion function for handwriting.

[0012] FIG. 5 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0013] FIG. 6 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0014] FIG. 7 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0015] FIG. 8 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0016] FIG. 9 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to obtain information about an angle at which a line is inclined with respect to a reference line and information about an angle at which text is inclined with respect to the reference line.

[0017] Fig. 10 is a diagram showing an example of feature information according to one embodiment.

[0018] FIG. 11 is a drawing illustrating strokes to explain an operation of an electronic device according to one embodiment to obtain information on the writing speed of a handwriting input.

[0019] Fig. 12 is a diagram showing an example of feature information according to one embodiment.

[0020] Fig. 13 is a diagram showing the result of performing an auto-completion function according to one embodiment.

[0021] Fig. 14 is a diagram showing the result of performing an auto-completion function according to one embodiment.

[0022] Fig. 15 is a diagram for explaining a generative artificial intelligence model according to one embodiment.

[0023] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0024] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0025] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0026] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0027] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0030] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0031] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0032] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0036] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0038] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0041] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0042] In one embodiment, the antenna module (197) may generate a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0043] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0046] Referring to FIG. 2, according to one embodiment, an electronic device (201) (e.g., electronic device (101) of FIG. 1) may include a processor (220) (e.g., processor (120) of FIG. 1), a memory (230) (e.g., memory (130) of FIG. 1), and a touch screen (260) (e.g., display (160) of FIG. 1).

[0047] According to one embodiment, the electronic device (201) may be implemented identically or similarly to the electronic device (101) of FIG. 1. For example, the electronic device (201) may be implemented as a smart phone or a tablet PC.

[0048] According to one embodiment, the processor (220) can control the overall operation of the electronic device (201). According to one embodiment, the processor (220) can be implemented identically or similarly to the processor (120) of FIG. 1.

[0049] According to one embodiment, the memory (230) may store instructions that cause the electronic device (201) to perform operations.

[0050] According to one embodiment, the memory (230) can store a generative AI model (340 of FIG. 3). According to one embodiment, the generative AI model (340) may be stored in a server or an external device. For example, the generative AI model (340) can use existing content, such as text, audio, and / or images, to newly generate content similar to the existing content. For example, the generative AI model (340) can learn patterns of content and generate new content as an inference result. For example, the generative AI model (340) can perform at least one of an in-painting operation or an out-painting operation on an image to generate (or obtain, output) a new image. Alternatively, the generative AI model (340) can receive an image corresponding to a user's handwriting input and generate (or obtain, output) a new image that has at least a partially modified shape of the handwriting input.

[0051] According to one embodiment, the processor (220) may obtain a plurality of automatically completed texts from text representing an incomplete word or an incomplete sentence using the generative AI model (340). According to one embodiment, the processor (220) may obtain a plurality of handwritings that have been transformed from a plurality of texts based on feature information about handwriting input using the generative AI model (340). According to one embodiment, the generative AI model (340) may include an artificial intelligence model that has learned information about the handwriting of the handwriting data by using handwriting data including handwriting inputs acquired through the touch screen (260) as learning data. According to one embodiment, the generative AI model (340) may include an artificial intelligence model that has learned information about the handwriting of the handwriting data.

[0052] According to one embodiment, the processor (220) may execute an auto-completion function for handwriting input obtained through the touch screen (260).

[0053] According to one embodiment, the processor (220) may acquire a first handwriting input including a plurality of strokes via the touch screen (260). According to one embodiment, the first handwriting input may be a handwriting input including an incomplete word or a handwriting input including an incomplete sentence. According to one embodiment, the first handwriting input may represent handwriting. For example, the first handwriting input may be acquired using a stylus pen.

[0054] According to one embodiment, the handwriting input may include a stroke. According to one embodiment, the stroke may represent a continuous stroke from the point where the handwriting input starts to the point where the handwriting input is released. According to one embodiment, the stroke may include at least one point. According to one embodiment, the processor (220) may acquire a point at a specified time interval at a location where the stylus pen and the touch screen (260) come into contact. According to one embodiment, the processor (220) may also acquire information about the order and coordinates in which at least one point included in the stroke is acquired.

[0055] In one embodiment, the processor (220) can determine whether the first handwriting input is text. For example, the processor (220) can use an artificial intelligence model stored in the memory (230) to determine whether the first handwriting input is text. The artificial intelligence model can be implemented as a document layout analyzer. However, this is merely an example, and the processor (220) can also use a separate algorithm to determine whether the first handwriting input is text.

[0056] According to one embodiment, if the processor (220) determines that the first handwriting input is text, the processor (220) may determine the number of syllables of the first handwriting input. According to one embodiment, if the number of syllables is determined to be greater than a specified number, the processor (220) may obtain coordinate information and feature information of the first handwriting input. According to one embodiment, if the number of syllables is determined to be not greater than the specified number, the processor (220) may not obtain the coordinate information and feature information of the first handwriting input. For example, the specified number may indicate the number of syllables corresponding to the handwriting input for which an auto-completion function for the handwriting input is executed. The specified number may be set by the user or by the processor (220). For example, the specified number may be 2. According to one embodiment, the processor (220) may obtain coordinate information and feature information of the first handwriting input regardless of the number of syllables.

[0057] According to one embodiment, the processor (220) may obtain coordinate information of points of a plurality of strokes included in a first handwriting input at specified times. For example, the processor (220) may obtain coordinate information of points of a plurality of strokes at specified times through a sensor. For example, the processor (220) may obtain coordinates of a first point among a plurality of points included in a plurality of strokes at a first time. The processor (220) may obtain coordinates of a second point among a plurality of points included in a plurality of strokes, which is obtained after the first point, at a second time after a specified time from the first time. The specified time may be automatically set by the processor (220) or may be set by a user.

[0058] According to one embodiment, the processor (220) may obtain handwriting speed information of a plurality of strokes included in a first handwriting input as characteristic information of the first handwriting input. According to one embodiment, the processor (220) may check the distance between coordinates of points of a plurality of strokes included in the first handwriting input obtained at specified times. According to one embodiment, the processor (220) may obtain handwriting speed information of a plurality of strokes based on the distance between coordinates of points of a plurality of strokes included in the first handwriting input obtained at specified times. For example, the processor (220) may obtain handwriting speed information of a stroke connecting the first point and the second point based on the coordinates of a first point obtained at a first time, the coordinates of a second point obtained at a second time after a specified time from the first time, and the specified time.

[0059] According to one embodiment, the processor (220) may obtain at least one of the number of strokes included in the first handwriting input, the number of syllables corresponding to the first handwriting input, or the shapes of the plurality of strokes as feature information.

[0060] According to one embodiment, information about the shape of the plurality of strokes may include information about a portion at which each of the texts corresponding to the syllables of the first handwriting input is connected to each other. According to one embodiment, the processor (220) may determine whether each of the texts corresponding to the syllables of the first handwriting input is connected to each other based on coordinate information of points of the plurality of strokes. For example, the processor (220) may determine whether a first text corresponding to a first syllable of the first handwriting input and a second text corresponding to a second syllable of the first handwriting input are connected. If at least one stroke among the strokes included in the first text and at least one stroke among the strokes included in the second text are connected to each other, it may be determined that the first text and the second text are connected.

[0061] According to one embodiment, information regarding the shapes of the plurality of strokes may include information regarding the portions of each of the plurality of strokes that are connected to each other. According to one embodiment, the processor (220) may determine whether each of the plurality of strokes is connected to each other based on coordinate information of points of the plurality of strokes.

[0062] In one embodiment, information about the shape of a plurality of strokes may include information about a portion of the plurality of strokes where a serif is identified. For example, a serif may indicate a curved portion of a stroke.

[0063] According to one embodiment, the processor (220) can identify the line of the first handwriting input. For example, the processor (220) can identify the line of the first handwriting input using an artificial intelligence model stored in the memory (230). The artificial intelligence model can be implemented as a document layout analyzer. However, this is an example, and the processor (220) can identify the line of the first handwriting input using a separate algorithm. According to one embodiment, the processor (220) can obtain information indicating an angle at which the line of the first handwriting input is inclined with respect to a reference line as feature information. For example, the reference line can be automatically set by the processor (220) or set by the user. For example, the angle at which the line of the first handwriting input is inclined with respect to the reference line can indicate a skew angle.

[0064] According to one embodiment, the processor (220) may obtain, as feature information, information indicating the angle at which each text representing a syllable corresponding to the first handwriting input is inclined with respect to a baseline. At this time, the processor (220) may obtain an average value of the angles at which the syllables in the first handwriting input are inclined with respect to the baseline. The processor (220) may obtain, as feature information, information indicating the difference between the average value and each of the angles. For example, the angle at which each text representing a syllable is inclined with respect to the baseline may indicate a slant angle.

[0065] Depending on the implementation, according to one embodiment, the processor (220) may further obtain, as feature information, the difference between the angle at which the line of the first handwriting input is inclined with respect to the baseline and the angle at which the text of the first handwriting input is inclined with respect to the baseline.

[0066] According to one embodiment, the processor (220) inputs coordinate information and feature information of the first handwriting input to the generative AI model (340), thereby obtaining a plurality of handwritings in which a plurality of texts are automatically completed from at least one text corresponding to the first handwriting input and are transformed based on the feature information. According to one embodiment, the processor (220) may input the first handwriting input to the generative AI model (340) instead of the coordinate information of the first handwriting input. Depending on the implementation, the electronic device (201) may input the first handwriting input to the generative AI model (340) in addition to the coordinate information and the feature information.

[0067] According to one embodiment, the processor (220) may obtain a plurality of automatically completed texts, at least one of which corresponds to a first handwriting input. For example, the processor (220) may obtain a plurality of automatically completed texts based on history information of handwriting inputs obtained prior to obtaining the first handwriting input and contextual information of the first handwriting input.

[0068] According to one embodiment, the processor (220) may obtain a plurality of handwritings that have been transformed from a plurality of texts based on feature information. For example, the processor (220) may obtain a plurality of handwritings that have feature information identical to or similar to the feature information of the first handwriting input.

[0069] For example, the first handwritten input may indicate "Hello." For example, a plurality of texts automatically completed with at least one text corresponding to the first handwritten input may include "sseumnida," "da," and "ne." For example, the plurality of handwritten inputs may include a first handwritten input in which "seumnida" is transformed based on feature information, a second handwritten input in which "da" is transformed based on feature information, and a third handwritten input in which "ne" is transformed based on feature information.

[0070] For example, the angles at which the lines of the first, second, and third handwritings are inclined with respect to the baseline may be the same as the angles at which the lines of the first handwriting input are inclined with respect to the baseline. For example, the angles at which the text representing the syllable corresponding to the first handwriting, the text representing the syllable corresponding to the second handwriting, and the text representing the syllable corresponding to the third handwriting are inclined with respect to the baseline may be the same as the angles at which the text representing the syllables of the first handwriting input are inclined with respect to the baseline. For example, the handwriting of the first, second, and third handwritings may be the same as or similar to the handwriting of the first handwriting input.

[0071] According to one embodiment, the processor (220) can display a first handwriting input and a plurality of handwritings on a touch screen (260).

[0072] Accordingly, according to one embodiment, the electronic device (201) can obtain a completed text from a handwriting input representing an incomplete text input through a touch screen (260), thereby providing a convenient handwriting environment to the user. According to one embodiment, the electronic device (201) can obtain a handwriting that has modified the completed text based on the characteristics of the incomplete text. Accordingly, the characteristics of the handwriting that has modified the completed text can have characteristics that are identical or similar to the characteristics of the incomplete text.

[0073] FIG. 3 is a schematic block diagram of a memory according to one embodiment.

[0074] Referring to FIG. 3, according to one embodiment, the memory (230) (e.g., the memory (230) of FIG. 2) may include a text recognition module (310), a text analysis module (320), and a generative AI model (340). According to one embodiment, the generative AI model (340) may include an auto-completion module (342), and a transformation module (343). According to one embodiment, at least some of the text recognition module (310), the text analysis module (320), and the generative AI model (340) may be implemented in hardware.

[0075] According to one embodiment, the processor (220) (e.g., the processor (220) of FIG. 2) can obtain a first handwriting input via a touch screen (260) (e.g., the touch screen (260) of FIG. 2).

[0076] In one embodiment, the text recognition module (310) may determine that the first handwriting input is text. For example, the text recognition module (310) may include an artificial intelligence model implemented as a document layout analyzer.

[0077] According to one embodiment, if the text recognition module (310) determines that the first handwriting input is text, it can check the number of syllables corresponding to the first handwriting input. According to one embodiment, the text recognition module (310) can check whether the number of syllables corresponding to the first handwriting input is greater than a specified number. For example, the specified number may indicate the number of syllables corresponding to the handwriting input for which an auto-completion function for the handwriting input is executed.

[0078] According to one embodiment, the text analysis module (320) can analyze the first handwriting input. According to one embodiment, the text analysis module (320) can obtain coordinate information and feature information of the first handwriting input based on the results of analyzing the first handwriting input.

[0079] According to one embodiment, the text analysis module (320) can obtain coordinate information of each point included in a plurality of strokes included in the first handwriting input at specified intervals.

[0080] According to one embodiment, the handwriting speed analysis module (330) may obtain handwriting speed information of a plurality of strokes included in a first handwriting input as characteristic information of the first handwriting input. According to one embodiment, the handwriting speed analysis module (330) may obtain handwriting speed information of a plurality of strokes based on the distance between coordinates of points of the plurality of strokes.

[0081] According to one embodiment, the text analysis module (320) may obtain at least one of the number of strokes included in the first handwriting input, the number of syllables corresponding to the first handwriting input, or the shape of the plurality of strokes as feature information.

[0082] According to one embodiment, information about the shapes of the plurality of strokes may include information about the portions of texts corresponding to syllables of the first handwriting input that are connected to each other. According to one embodiment, the text analysis module (320) may determine whether each of the texts corresponding to syllables of the first handwriting input is connected to each other based on coordinate information of points of the plurality of strokes.

[0083] According to one embodiment, information about the shapes of the plurality of strokes may include information about the portions where each of the plurality of strokes is connected to each other. According to one embodiment, the text analysis module (320) may obtain information about the portions where each of the plurality of strokes is connected to each other based on coordinate information of the points of the plurality of strokes.

[0084] According to one embodiment, information about the shape of a plurality of strokes may include information about the portion of the strokes where a serif is identified. For example, a serif may indicate a curved portion of the stroke. According to one embodiment, the text analysis module (320) may obtain information about the portion of the strokes where a serif is identified.

[0085] According to one embodiment, the text analysis module (320) can identify the line of the first handwriting input. According to one embodiment, the text analysis module (320) can obtain information indicating the angle at which the line of the first handwriting input is inclined with respect to a baseline as feature information of the first handwriting input. According to one embodiment, the text analysis module (320) can obtain information indicating the angle at which each text representing a syllable corresponding to the first handwriting input is inclined with respect to a baseline as feature information.

[0086] According to one embodiment, the text analysis module (320) may obtain an average value of the angles at which syllables in the first handwriting input are inclined with respect to a baseline. The text analysis module (320) may further obtain information representing the difference between the average value and each of the angles as feature information. According to one embodiment, the text analysis module (320) may further obtain information representing the difference between the angle at which a line in the first handwriting input is inclined with respect to a baseline and the angle at which each text representing a syllable is inclined with respect to the baseline as feature information.

[0087] According to one embodiment, the processor (220) may input coordinate information and feature information of the first handwriting input into the generative AI model (340). According to one embodiment, the generative AI model (340) is illustrated as being stored in the memory (230), but the generative AI model (340) may also be stored in a server or an external device.

[0088] According to one embodiment, the auto-completion module (342) can obtain a plurality of texts in which text corresponding to the first handwriting input is automatically completed.

[0089] According to one embodiment, the transformation module (343) can obtain a plurality of handwritings that have transformed a plurality of texts based on feature information.

[0090] For example, the transformation module (343) can transform the angle at which each of the plurality of handwritings is tilted with respect to a baseline so that the angle at which the line of the first handwriting input is tilted with respect to the baseline is the same as or similar to the angle at which the line of the first handwriting input is tilted with respect to the baseline. For example, the transformation module (343) can transform the angle at which each of the texts included in the plurality of handwritings is tilted with respect to a baseline so that the angle at which the text representing the syllable of the first handwriting input is tilted with respect to the baseline is the same as or similar to the angle at which the text representing the syllable of the first handwriting input is tilted with respect to the baseline. For example, the transformation module (343) can transform the handwriting of the plurality of handwritings so that the handwriting is the same as or similar to the handwriting of the first handwriting input. However, this is an example, and the transformation module (343) according to an embodiment can obtain a plurality of handwritings that have transformed a plurality of texts in various ways. FIG. 4 is a flowchart illustrating an operation of an electronic device according to an embodiment of the present invention to execute an automatic completion function for handwriting.

[0091] Referring to FIG. 4, according to one embodiment, in operation 411, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may execute an auto-completion function for handwriting input obtained through a touch screen (260) (e.g., the touch screen (260) of FIG. 2). According to one embodiment, when the auto-completion function is executed, the electronic device (201) may obtain a completed sentence or a completed word from an incomplete sentence or an incomplete word.

[0092] According to one embodiment, in operation 413, the electronic device (201) may acquire a first handwriting input including a plurality of strokes through the touch screen (260) after the auto-completion function is executed. For example, the first handwriting input may be acquired by a stylus pen. According to one embodiment, the first handwriting input may be a handwriting input including an incomplete word or may represent a handwriting input including an incomplete sentence. According to one embodiment, the first handwriting input may represent handwriting.

[0093] According to one embodiment, the electronic device (201) can determine whether the first handwriting input is text. According to one embodiment, if the electronic device (201) determines that the first handwriting input is text, the electronic device (201) can determine the number of syllables of the first handwriting input. According to one embodiment, if the number of syllables is determined to be greater than a specified number, the electronic device (201) can obtain coordinate information and feature information of the first handwriting input. For example, the specified number may indicate the number of syllables of the handwriting input for which an auto-completion function for the handwriting input is executed. According to one embodiment, the electronic device (201) may obtain coordinate information and feature information of the first handwriting input regardless of the number of syllables.

[0094] According to one embodiment, in operation 415, the electronic device (201) may obtain coordinate information of a plurality of strokes. According to one embodiment, the electronic device (201) may obtain coordinate information of a plurality of strokes at a specified time interval using a sensor included in the electronic device (201). For example, the electronic device (201) may obtain coordinates of a first point among a plurality of points included in the plurality of strokes at a first time interval. The electronic device (201) may obtain coordinates of a second point among a plurality of points included in the plurality of strokes, which is obtained after the first point, at a second time interval after a specified time interval from the first time interval.

[0095] According to one embodiment, in operation 417, the electronic device (201) may obtain feature information associated with a plurality of strokes.

[0096] According to one embodiment, the electronic device (201) may obtain at least one of the number of strokes included in the first handwriting input, the number of syllables corresponding to the first handwriting input, or the shapes of the plurality of strokes as feature information. According to one embodiment, the electronic device (201) may obtain handwriting speed information of the plurality of strokes included in the first handwriting input as feature information of the first handwriting input. According to one embodiment, the electronic device (201) may obtain at least one of information on the angle at which a line of the first handwriting input is inclined with respect to a baseline or information on the angle at which texts corresponding to syllables of the first handwriting input are inclined with respect to a baseline as feature information.

[0097] According to one embodiment, in operation 419, the electronic device (201) inputs coordinate information and feature information into a generative AI model (340) (e.g., the generative AI model (340) of FIG. 3) stored in a memory (230) (e.g., the memory (230) of FIG. 2) to obtain a plurality of handwritings in which a plurality of texts are transformed and automatically completed from at least one text corresponding to the first handwriting input. According to one embodiment, the electronic device (201) may input the first handwriting input into the generative AI model (340) instead of the coordinate information. Depending on the implementation, the electronic device (201) may input the first handwriting input into the generative AI model (340) in addition to the coordinate information and the feature information.

[0098] According to one embodiment, the electronic device (201) may obtain a plurality of automatically completed texts, at least one of which corresponds to a first handwriting input, based on coordinate information. According to one embodiment, the electronic device (201) may obtain a plurality of automatically completed texts based on history information of handwriting inputs obtained prior to obtaining the first handwriting input and context information of the first handwriting input.

[0099] According to one embodiment, the electronic device (201) may obtain a plurality of handwritings that have been transformed from a plurality of texts based on the characteristic information of the first handwriting input. According to one embodiment, the electronic device (201) may obtain a plurality of handwritings that have characteristic information that is identical to or similar to the characteristic information of the first handwriting input.

[0100] According to one embodiment, in operation 421, the electronic device (201) may display a first handwriting input and a plurality of handwritings on a touch screen (260).

[0101] Accordingly, according to one embodiment, the electronic device (201) can obtain a completed text from a handwriting input representing an incomplete text input through a touch screen (260), thereby providing a convenient handwriting environment to the user. According to one embodiment, the electronic device (201) can obtain a handwriting that has modified the completed text based on the characteristics of the incomplete text. Accordingly, the characteristics of the handwriting that has modified the completed text can have characteristics that are identical or similar to the characteristics of the incomplete text.

[0102] FIG. 5 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0103] Referring to FIG. 5, according to one embodiment, in operation 511, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a first handwriting input through a touch screen (260) (e.g., the touch screen (260) of FIG. 2).

[0104] According to one embodiment, in operation 513, the electronic device (201) may verify the lines of the first handwriting input. For example, the electronic device (201) may verify the lines of the first handwriting input using an artificial intelligence model implemented as a document layout analyzer stored in the memory (230) (e.g., the memory (230) of FIG. 2).

[0105] According to one embodiment, in operation 515, the electronic device (201) may obtain information indicating an angle at which a line is tilted with respect to a reference line as feature information of the first handwriting input. For example, the angle at which a line of the first handwriting input is tilted with respect to the reference line may indicate a skew angle.

[0106] FIG. 6 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0107] Referring to FIG. 6, according to one embodiment, in operation 611, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a first handwriting input through a touch screen (260) (e.g., the touch screen (260) of FIG. 2).

[0108] According to one embodiment, in operation 613, the electronic device (201) may identify syllables of the first handwriting input. For example, the electronic device (201) may identify syllables of the first handwriting input using an artificial intelligence model implemented as a document layout analyzer stored in a memory (230) (e.g., memory (230) of FIG. 2 ).

[0109] According to one embodiment, in operation 615, the electronic device (201) may obtain information on the angle at which texts corresponding to syllables are tilted relative to a baseline as feature information. For example, the angle at which texts corresponding to syllables of the first handwriting input are tilted relative to a baseline may represent a slant angle.

[0110] According to one embodiment, the electronic device (201) can obtain an average value for angles at which texts are tilted relative to a baseline. According to one embodiment, the electronic device (201) can further obtain values ​​representing the difference between the average value and each of the angles as feature information.

[0111] According to one embodiment, the electronic device (201) may further obtain, as feature information, the difference between the angle at which the line of the first handwriting input is inclined with respect to the baseline and the angle at which the texts corresponding to the syllables are inclined with respect to the baseline.

[0112] FIG. 7 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0113] Referring to FIG. 7, according to one embodiment, in operation 711, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may check the coordinates of points of a plurality of strokes at specified times. According to one embodiment, the stroke may include at least one point. According to one embodiment, the electronic device (201) may obtain a point at specified times at a location where the stylus pen and the touch screen (260) come into contact. According to one embodiment, the electronic device (201) may check the coordinates of the obtained point through a sensor.

[0114] According to one embodiment, in operation 713, the electronic device (201) can determine the distance between coordinates. For example, the electronic device (201) can determine the distance between coordinates of two adjacent points among points of a plurality of strokes.

[0115] According to one embodiment, in operation 715, the electronic device (201) may obtain stroke speed information of a plurality of strokes as feature information. According to one embodiment, the electronic device (201) may obtain stroke speed information of a plurality of strokes based on the distance between coordinates of two adjacent points among the points of the plurality of strokes and the time at which the coordinates of the two adjacent points were obtained.

[0116] FIG. 8 is a flowchart illustrating an operation of an electronic device according to one embodiment to acquire feature information.

[0117] Referring to FIG. 8, according to one embodiment, in operation 811, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may check the number of multiple strokes included in the first handwriting input. According to one embodiment, the stroke may represent a continuous stroke from a point where the handwriting input starts to a point where the handwriting input is released.

[0118] According to one embodiment, in operation 813, the electronic device (201) can determine the number of syllables included in the plurality of strokes. According to one embodiment, the electronic device (201) can distinguish the plurality of strokes on a syllable basis. According to one embodiment, the electronic device (201) can determine the number of syllables based on the plurality of strokes distinguished on a syllable basis.

[0119] According to one embodiment, in operation 815, the electronic device (201) can identify the shape of a plurality of strokes.

[0120] According to one embodiment, information about the shape of the plurality of strokes may include information about the portions of each of the plurality of strokes that are connected to each other.

[0121] For example, the electronic device (201) can identify the portion where each of the plurality of strokes is connected (overlapped) to each other. The electronic device (201) can identify the portion where each of the plurality of strokes is connected (overlapped) to each other based on coordinate information of points of the plurality of strokes.

[0122] According to one embodiment, information about the shape of the plurality of strokes may include information about the portion where each of the texts corresponding to the syllables of the first handwriting input is connected to each other. For example, the electronic device (201) may, based on the plurality of strokes distinguished by syllable units, determine the portion where each of the texts corresponding to the syllables is connected to each other (overlapping). Based on coordinate information of points of the plurality of strokes, the electronic device (201) may determine whether each of the texts corresponding to the syllables of the first handwriting input is connected to each other.

[0123] In one embodiment, information about the shape of a plurality of strokes may include information about a portion of the plurality of strokes where a serif is identified. For example, a serif may indicate a curved portion of a stroke.

[0124] According to one embodiment, in operation 817, the electronic device (201) may obtain at least one of the number of the plurality of strokes, the number of syllables included in the plurality of strokes, or the shape of the plurality of strokes as feature information.

[0125] FIG. 9 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to obtain information about the angle at which a line is tilted relative to a reference line and information about the angle at which text is tilted relative to the reference line. FIG. 10 is a diagram illustrating an example of feature information according to one embodiment of the present invention.

[0126] Referring to FIGS. 9 and 10 , according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2 ) may obtain a first handwriting input (900) through a touch screen (260) (e.g., the touch screen (260) of FIG. 2 ). For example, the first handwriting input (900) may indicate “nice to meet you.”

[0127] According to one embodiment, the electronic device (201) can identify a line (920) of a first handwriting input (900). According to one embodiment, the electronic device (201) can obtain an angle (a) between the line (920) and a reference line (910) as feature information of the first handwriting input (900).

[0128] According to one embodiment, the electronic device (201) can confirm texts corresponding to syllables of the first handwriting input (900). According to one embodiment, the electronic device (201) can confirm that the texts corresponding to the syllables are “반” and “갑”. According to one embodiment, the electronic device (201) can obtain an angle (b) between the text (e.g., 반) and the baseline (910) as feature information of the first handwriting input (900). For example, the angle (b) may represent 106 degrees. According to one embodiment, the electronic device (201) can obtain an angle (c) between the text (e.g., 갑) and the baseline (910) as feature information of the first handwriting input (900). For example, the angle (c) may represent 104 degrees.

[0129] According to one embodiment, the electronic device (201) may obtain the deviation as feature information of the first handwriting input (900). According to one embodiment, the deviation may represent a difference value of the angle between each text included in the handwriting input and the reference line from an average value of the angles between the texts included in the handwriting input and the reference line. According to one embodiment, the electronic device (201) may determine that the average value of the angles between the texts included in the handwriting input and the reference line is 105 degrees. According to one embodiment, the electronic device (201) may determine that the difference between the angle (e.g., 106) between the text indicating “half” and the reference line (910) and the average value (e.g., 105) is 1. According to one embodiment, the electronic device (201) may determine that the difference between the average value (e.g., 105) and the angle (e.g., 104) between the text indicating “gap” and the reference line (910) is 1.

[0130] FIG. 11 is a drawing illustrating strokes to explain an operation of an electronic device according to one embodiment to obtain information on the writing speed of a handwriting input.

[0131] Referring to FIG. 11, according to one embodiment, a plurality of strokes of the first handwriting input (900 of FIG. 9) may include a plurality of points. According to one embodiment, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain coordinate information of each of the plurality of points at a specified time interval. According to one embodiment, the electronic device (201) may obtain handwriting speed information of the plurality of strokes based on the distance between the plurality of points and the specified time interval.

[0132] For example, according to one embodiment, the electronic device (201) can obtain a first point (1111) included in a first stroke (1110) at a first time. The electronic device (201) can obtain a second point (1112) after the first point (1111) at a second time, which is a specified time after the first time. According to one embodiment, the electronic device (201) can obtain pen-speed information for a portion connecting the first point (1111) and the second point (1112) among the first strokes (1110) based on the specified time, the coordinates corresponding to the first point (1111), and the coordinates corresponding to the second point (1112).

[0133] Fig. 12 is a diagram showing an example of feature information according to one embodiment.

[0134] Referring to FIG. 12, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) can identify syllables of a first handwriting input (900 of FIG. 9). According to one embodiment, the electronic device (201) can identify the number of texts corresponding to the syllables of the first handwriting input (900).

[0135] For example, the electronic device (201) can identify “반” corresponding to the first syllable of the first handwriting input (900) and “갑” corresponding to the second syllable. The electronic device (201) can identify that the number of texts corresponding to the syllables of the first handwriting input (900) is two.

[0136] According to one embodiment, the electronic device (201) can check the number of multiple strokes included in the first handwriting input (900). For example, the electronic device (201) can check the number of multiple strokes included in the first handwriting input (900) as 9.

[0137] According to one embodiment, the electronic device (201) can check whether texts corresponding to syllables of the first handwriting input (900) are connected to each other. For example, the electronic device (201) can check that the strokes included in “반” corresponding to the first syllable of the first handwriting input (900) and the strokes included in “갑” corresponding to the second syllable of the first handwriting input (900) are not connected to each other. The electronic device (201) can check that the number of connections between texts of the first handwriting input (900) is 0.

[0138] According to one embodiment, the electronic device (201) can check the number of cursive characters of consonants and vowels included in texts corresponding to syllables of the first handwriting input (900). For example, cursive characters may indicate a state in which a portion of a stroke is curved. The state in which a portion of a stroke is curved may include a portion in which a serif is identified. For example, the electronic device (201) can check a portion (1201) in which a cursive character is identified among a plurality of strokes included in the first handwriting input (900). The electronic device (201) can check that the number of portions in which a cursive character is identified among a plurality of strokes included in the first handwriting input (900) is 1.

[0139] Fig. 13 is a diagram showing the result of performing an auto-completion function according to one embodiment.

[0140] Referring to FIG. 13, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input coordinate information and feature information of a first handwriting input (900) into a generative AI model (340) (e.g., the generative AI model (340) of FIG. 3).

[0141] According to one embodiment, the electronic device (201) can obtain a plurality of texts that are automatically completed based on the coordinate information of the first handwriting input (900) using the generative AI model (340). For example, the text of the first handwriting input (900) can represent "nice to meet you." For example, the plurality of texts can represent "seumnida", "da", and "ne".

[0142] According to one embodiment, the electronic device (201) can obtain multiple texts in which texts are automatically completed and transformed into handwritings (1310, 1320, 1330) based on feature information using a generative AI model (340).

[0143] According to one embodiment, the electronic device (201) can display handwritings (1310, 1320, 1330) through a touch screen (260) (e.g., the touch screen (260) of FIG. 2).

[0144] According to one embodiment, the electronic device (201) may display a recommended text (1301) that automatically completes the first handwriting input (900) through the touch screen (260). According to one embodiment, the first handwriting input (900) and the recommended text (1301) may represent handwriting (1310).

[0145] According to one embodiment, the electronic device (201) may apply a visual effect to the recommended text (1301) so that the recommended text (1301) is distinguished from the first handwriting input (900). For example, the electronic device (201) may display the recommended text (1301) in a blurred manner. For example, the electronic device (201) may apply a color of the recommended text (1301) that is different from the color of the first handwriting input (900).

[0146] According to one embodiment, the electronic device (201) may terminate the auto-completion function when an input for any one of the handwritings (1310, 1320, 1330) is confirmed.

[0147] Fig. 14 is a diagram showing the result of performing an auto-completion function according to one embodiment.

[0148] Referring to FIGS. 13 and 14, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) can further obtain multiple texts in which texts are automatically completed using a generative AI model (340) (e.g., the generative AI model (340) of FIG. 3) and transformed handwritings (1410, 1420, 1430, 1440, 1450, 1460) based on feature information.

[0149] According to one embodiment, the handwritings (1410, 1420) may represent handwriting that is different from the first text (1310) that automatically completed the first handwriting input (900) in handwriting style and / or feature information (e.g., number of cursive characters, number of connections between texts, angle of inclination relative to a baseline, etc.).

[0150] According to one embodiment, the handwritings (1430, 1440) may represent handwriting that is different from the second text (1320) that automatically completed the first handwriting input (900) in handwriting style and / or feature information (e.g., number of cursive characters, number of connections between texts, angle of inclination relative to a baseline, etc.).

[0151] According to one embodiment, the handwritings (1450, 1460) may represent handwriting that is different from the third text (1330) that automatically completed the first handwriting input (900) in handwriting style and / or feature information (e.g., number of cursive characters, number of connections between texts, angle of inclination relative to a baseline, etc.).

[0152] Fig. 15 is a diagram for explaining a generative artificial intelligence model according to one embodiment.

[0153] Referring to FIG. 15, a user query / response interface (1510), an application / service component (1530), a knowledge repository (1520), an AI framework (1540), and a generative AI model (1560) (e.g., the generative AI model (340) of FIG. 3) may be stored in a memory (230) (e.g., the memory (230) of FIG. 2) or stored in a separate server. At least some of the user query / response interface (1510), the application / service component (1530), the knowledge repository (1520), the AI ​​framework (1540), or the generative AI model (1560) may be implemented in software or hardware.

[0154] According to one embodiment, a user query / response interface (1510) may receive a user's input. The user's input may be in the form of natural language, images, and / or videos, but is not limited thereto. Furthermore, context information may also be transmitted when the user's input is transmitted. The context information may include various additional information at the time of the user input. For example, the additional information may include information about the application currently being used by the user or information about the user's location. Furthermore, the user's input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user's input may also be in a non-natural language form, such as selecting a menu. The user query / response interface (1510) may output the results of a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The user query / response interface (1510) may output the results of a generative artificial intelligence system to the user. The output can be in natural language form, in the form of specific content, or in the form of actions requested by the user.

[0155] The AI ​​framework (1540) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.

[0156] User input received from the user query / response interface (1510) can be transmitted to a prompt design component (1541). The prompt design component (1541) can be used to generate prompts suitable for inputting user input into a large language model (LLM) or a large multimodal model (LMM). The prompt design component (1541) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (1541) can access a knowledge component including user preference data, a prompt library, and prompt examples based on the user input to generate prompts, and can transmit the generated prompts to the large language model (LLM) or the large multimodal model (LMM).

[0157] The API / Plug-in management component (1542) can communicate with external information when there is a request for additional information when passing user input as input to a generative model. The API / Plug-in management component (1542) can establish a channel for communicating with the outside of the AI ​​Interface through the API, and can enable access to various data sources (e.g., knowledge repositories (1520)) through the established channel. In addition, the API / Plug-in management component (1542) can request the application / service component (1530) through the API for an action that ultimately performs the user input, rather than an intermediate result, when the action needs to be performed in the application or service. Information obtained from the outside can be used to generate a prompt in the prompt design component (1541) together with the user input, or can be passed as input to the generative model.

[0158] The output modification component (also called a refiner component) (1543) can fine-tune the output from the generative model. For example, the output modification component (1543) can verify that the content generated through the LLM and / or LMM is not irrelevant, does not contain biased content, or does not contain harmful content. In addition, the output modification component (1543) can determine to what extent the content matches the user's desired result and, if necessary, can perform additional processing. The output modification component (1543) can additionally configure and provide the user with hints to avoid unwanted output.

[0159] A generative AI model (1560) can generally refer to an artificial intelligence neural network that creates new types of data based on user input information. A generative AI model (1560) can include an image-generating model and / or a language-generating model. Representative models for generating images include a generative adversarial network (GAN) and a variational auto encoder (VAE), and examples include a VAE and a Diffusion-based generative model that uses a Transformer structure. A language-generating model is a model trained to statistically output the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there are also LMMs (large multimodal models) that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0160] According to one embodiment, an electronic device (201) (e.g., electronic device (201) of FIG. 2) may include a touch screen (260) (e.g., touch screen (260) of FIG. 2), a memory (230) for storing instructions (e.g., memory (230) of FIG. 2), and at least one processor (220) (e.g., processor (220) of FIG. 2).

[0161] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain a first handwriting input including a plurality of strokes through the touch screen (260).

[0162] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain coordinate information of the plurality of strokes.

[0163] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain feature information related to the plurality of strokes, wherein the feature information may include information indicating an angle at which the first handwriting input is inclined with respect to a reference line.

[0164] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to input the coordinate information and the feature information into a generative AI model, so that at least one text corresponding to the first handwriting input obtained based on the coordinate information is automatically completed, and a plurality of handwritings are obtained by transforming the plurality of handwritings based on the feature information.

[0165] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to display the first handwriting input and the plurality of handwritings on the touch screen (260).

[0166] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to identify a line of the first handwriting input and obtain information indicating an angle at which the line is inclined with respect to the reference line as the feature information.

[0167] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to identify at least one syllable corresponding to the first handwriting input and obtain, as the feature information, information indicating an angle at which the at least one text corresponding to the at least one syllable is inclined with respect to the reference line.

[0168] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain an average value of angles at which the first syllable and the second syllable included in the at least one syllable are inclined with respect to the reference line, and obtain information representing a difference between the average value and each of the angles as the feature information.

[0169] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to check coordinates of points of the plurality of strokes at designated intervals, and to obtain stroke speed information of the plurality of strokes checked based on the distance between the coordinates as the feature information.

[0170] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain at least one of the number of the plurality of strokes, the number of syllables corresponding to the first handwriting input, or the shape of the plurality of strokes as the feature information.

[0171] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to: determine the number of syllables of the plurality of texts based on determining that the first handwriting input represents a plurality of texts; and obtain the coordinate information and the feature information based on determining that the number of syllables is greater than a specified number.

[0172] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to input the first handwriting input instead of the coordinate information into the generative AI model (340) stored in the memory (230).

[0173] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain the plurality of texts in which the at least one text corresponding to the first handwriting input is automatically completed, and to obtain the plurality of handwritings in which the plurality of texts are transformed based on the feature information.

[0174] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to display the plurality of handwritings on the touch screen (260) as a result of performing the auto-completion function for the first handwriting input.

[0175] According to one embodiment, a method of operating an electronic device (201) may include an operation of obtaining a first handwriting input including a plurality of strokes through the touch screen (260).

[0176] According to one embodiment, the method of operating the electronic device (201) may include an operation of obtaining coordinate information of the plurality of strokes.

[0177] According to one embodiment, a method of operating an electronic device (201) includes an operation of obtaining feature information related to the plurality of strokes, wherein the feature information may include information indicating an angle at which the first handwriting input is inclined with respect to a reference line.

[0178] According to one embodiment, the operating method of the electronic device (201) may include an operation of inputting the coordinate information and the feature information into a generative AI model (340), and obtaining a plurality of texts in which at least one text corresponding to the first handwriting input obtained based on the coordinate information is automatically completed, and a plurality of handwritings in which the plurality of texts are transformed based on the feature information.

[0179] According to one embodiment, the method of operating the electronic device (201) may include an operation of displaying the first handwriting input and the plurality of handwritings on the touch screen (260).

[0180] According to one embodiment, the method of operating the electronic device (201) may include an operation of confirming a line of the first handwriting input.

[0181] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining information indicating an angle at which the line is inclined with respect to the reference line as the feature information.

[0182] According to one embodiment, the method of operating the electronic device (201) may include an operation of identifying at least one syllable corresponding to the first handwriting input.

[0183] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining, as the feature information, information on an angle at which the at least one text corresponding to the at least one syllable is tilted with respect to a baseline.

[0184] According to one embodiment, the method of operating the electronic device (201) may include an operation of obtaining an average value of angles at which the at least one syllable is inclined with respect to the reference line.

[0185] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining information representing the difference between the average value and each of the angles as the feature information.

[0186] According to one embodiment, the method of operating the electronic device (201) may include an operation of checking coordinates of points of the plurality of strokes at specified intervals.

[0187] According to one embodiment, the operating method of the electronic device (201) may include an operation of acquiring, as the feature information, the stroke speed information of the plurality of strokes identified based on the distance between the coordinates.

[0188] According to one embodiment, the operating method of the electronic device (201) may include an operation of acquiring at least one of the number of the plurality of strokes, the number of syllables included in the plurality of strokes, or the shape of the plurality of strokes as the feature information.

[0189] According to one embodiment, the method of operating the electronic device (201) may include an operation of checking the number of syllables of the plurality of texts based on checking that the first handwriting input represents the plurality of texts.

[0190] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining the coordinate information and the feature information based on determining that the number of syllables is greater than a specified number.

[0191] According to one embodiment, the method of operating the electronic device (201) may include an operation of inputting the first handwriting input into the generative AI model (340) stored in a memory (230) included in the electronic device (201).

[0192] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining the plurality of texts in which at least one text corresponding to the first handwriting input obtained based on the coordinate information is automatically completed.

[0193] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining the plurality of handwritings in which the plurality of texts are transformed based on the feature information.

[0194] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include obtaining a first handwriting input including a plurality of strokes through a touch screen (260) included in the electronic device (201).

[0195] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of obtaining coordinate information of the plurality of strokes.

[0196] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, the at least one operation including an operation of obtaining feature information related to the plurality of strokes, wherein the feature information may include information indicating an angle at which the first handwriting input is inclined with respect to a reference line.

[0197] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of inputting the coordinate information and the feature information into a generative AI model (340), thereby obtaining a plurality of texts in which at least one text corresponding to the first handwriting input obtained based on the coordinate information is automatically completed and a plurality of handwritings are transformed based on the feature information.

[0198] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of displaying the first handwriting input and the plurality of handwritings on the touch screen.

[0199] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0200] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0201] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0202] Various embodiments of the present document may be implemented as software (e.g., program (140)) including one or more commands stored in a storage medium (e.g., built-in memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101, 201)). For example, a processor (e.g., processor (120, 220)) of a machine (e.g., electronic device (101, 201)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the called at least one command. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means a device in which the storage medium is tangible, It simply means that it does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on a storage medium.

[0203] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0204] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), Touch screen (260); Memory (230) for storing instructions; and comprising at least one processor (220), The above instructions, when executed by the at least one processor, cause the electronic device to: Obtaining a first handwriting input including a plurality of strokes through the above touch screen, Obtain coordinate information of the above multiple strokes, Obtaining feature information related to the plurality of strokes, wherein the feature information includes information indicating an angle at which the first handwriting input is tilted with respect to a reference line, By providing the above coordinate information and the above feature information to the generative AI model (340), a plurality of texts automatically completed from at least one text corresponding to the first handwriting input are obtained by transforming a plurality of handwritings based on the feature information, An electronic device that displays the first handwriting input and the plurality of handwritings on the touch screen.

2. In paragraph 1, The above instructions, when executed by the at least one processor, cause the electronic device to: Check the line of the first handwriting input above, An electronic device that obtains information representing the angle at which the above line is inclined with respect to the above reference line as the feature information.

3. In any one of paragraphs 1 and 2, The above instructions, when executed by the at least one processor, cause the electronic device to: Identify at least one syllable corresponding to the first handwritten input, An electronic device that obtains, as feature information, information indicating an angle at which at least one text corresponding to at least one syllable is tilted with respect to the reference line.

4. In any one of paragraphs 1 to 3, The above instructions, when executed by the at least one processor, cause the electronic device to: The first syllable and the second syllable among the at least one syllable obtain an average value of the angles at which they are tilted with respect to the reference line, An electronic device that obtains information representing the difference between the average value and each of the angles as the feature information.

5. In any one of paragraphs 1 to 4, The above instructions, when executed by the at least one processor, cause the electronic device to: Check the coordinates of the points of the above multiple strokes at specified times, An electronic device that obtains stroke speed information of the plurality of strokes confirmed based on the distance between the above coordinates as the feature information.

6. In any one of paragraphs 1 to 5, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that obtains at least one of the number of the plurality of strokes, the number of syllables corresponding to the first handwriting input, or the shape of the plurality of strokes as the feature information.

7. In any one of paragraphs 1 to 6, The above instructions, when executed by the at least one processor, cause the electronic device to: Based on the determination that the above first handwriting input represents a plurality of texts, the number of syllables of the plurality of texts is determined, An electronic device that obtains the coordinate information and the feature information based on determining that the number of the above syllables is greater than a specified number.

8. In any one of paragraphs 1 to 7, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that provides the first handwriting input to the generative AI model stored in the memory.

9. In any one of paragraphs 1 to 8, The above instructions, when executed by the at least one processor, cause the electronic device to: Based on the above coordinate information, obtain the plurality of texts automatically completed from the at least one text corresponding to the first handwriting input, An electronic device that obtains a plurality of handwritings in which the plurality of texts are transformed based on the above characteristic information.

10. In any one of paragraphs 1 to 9, The above instructions, when executed by the at least one processor, cause the electronic device to: After the auto-completion function for handwriting input is executed, the first handwriting input is acquired, An electronic device that displays the plurality of handwritings on the touch screen as a result of performing the auto-completion function for the first handwriting input.

11. In the operating method of an electronic device (201), An action of obtaining a first handwriting input including a plurality of strokes through a touch screen (260) included in the electronic device; An operation of obtaining coordinate information of the above plurality of strokes; An operation of obtaining feature information related to the plurality of strokes, wherein the feature information includes information indicating an angle at which the first handwriting input is tilted with respect to a reference line, An operation of providing the above coordinate information and the above feature information to a generative AI model (340) to obtain a plurality of texts automatically completed from at least one text corresponding to the first handwriting input, and a plurality of handwritings transformed based on the feature information; and A method of operating an electronic device, comprising: inputting the first handwriting and displaying the plurality of handwritings on the touch screen.

12. A method of operating an electronic device, further comprising an operation according to any one of claims 2 to 10, in claim 11.

13. In a storage medium storing computer-readable instructions, the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device to perform at least one operation, the at least one operation being: An action of obtaining a first handwriting input including a plurality of strokes through a touch screen (260) included in the electronic device; An operation of obtaining coordinate information of the above plurality of strokes; An operation of obtaining feature information related to the plurality of strokes, wherein the feature information includes information indicating an angle at which the first handwriting input is tilted with respect to a reference line; An operation of providing the above coordinate information and the above feature information to a generative AI model (340) to obtain a plurality of texts automatically completed from at least one text corresponding to the first handwriting input, and a plurality of handwritings transformed based on the feature information; and A storage medium including the first handwriting input and the operation of displaying the plurality of handwritings on the touch screen.

Images