Method and electronic device for converting text into handwriting

Abstract

An electronic device according to one embodiment comprises: a communication module including at least one communication circuit; a display; memory for storing instructions; and a processor (120). The instructions, when executed by the processor, may instruct the electronic device to: collect handwriting samples of a user, input in handwriting, on the basis of at least one of the display or the memory; identify character characteristics of the collected handwriting samples; acquire a first cluster learning model similar to the character characteristics of the handwriting samples, among cluster learning AI models grouped by cluster-learning a plurality of pieces of handwriting data according to similar characteristics, through the communication module; configure a personalized database corresponding to the character characteristics of the user by using the handwriting samples and the first cluster learning model; receive text input through the display; and convert the text into handwriting on the display on the basis of the personalized handwriting database.

Description

Method for converting text to handwriting and electronic device

[0001] The present invention relates to a method for converting text into handwriting and an electronic device.

[0002] Electronic devices provide a function that allows users to take necessary notes anytime and anywhere without a notebook or pen. For example, users can write directly on a display (e.g., a touchscreen) using their hand or an electronic pen. The electronic device can receive the touch trajectory (or coordinates) of touching (or contacting) the display as input in the form of handwriting (or text). By inputting handwriting using an electronic pen, just as one would write on a notebook with a pen, users can conveniently create notes while experiencing an analog sensibility.

[0003] While the function of recognizing handwriting and converting it into text on electronic devices is supported in commercially available products, the function of converting text into the user's handwriting is not supported.

[0004] Various embodiments propose a device and method that support a function of converting text entered in a memo, note, calendar, messenger, or text input application into handwriting that reflects the user's handwriting characteristics.

[0005] However, the problems intended to be solved in this disclosure are not limited to those mentioned above, and may be expanded in various ways without departing from the spirit and scope of this disclosure.

[0006] An electronic device according to one embodiment may include a communication module comprising at least one communication circuit. An electronic device according to one embodiment may include a display. An electronic device according to one embodiment may include a memory for storing instructions. An electronic device according to one embodiment may include a processor (120). When the instructions according to one embodiment are executed by the processor, the electronic device may collect a user's handwriting sample entered in handwriting based on at least one of the display and the memory. The instructions according to one embodiment may identify the handwriting characteristics of the collected handwriting sample and, through the communication module, cluster learning a plurality of handwriting data according to similar characteristics to obtain a first cluster learning model among the grouped cluster learning AI models that is similar to the handwriting characteristics of the handwriting sample. The instructions according to one embodiment may construct a personalized database corresponding to the user's handwriting characteristics using the handwriting sample and the first cluster learning model. The instructions according to one embodiment may receive text input through the display. The instructions according to one embodiment may convert the text into handwriting on the display based on a personalized handwriting database.

[0007] A method for converting text of an electronic device into handwriting according to one embodiment may include an operation of collecting a user's handwriting sample entered as handwriting based on at least one of a display and a memory. A method according to one embodiment may include an operation of verifying the handwriting characteristics of the collected handwriting sample. A method according to one embodiment may include an operation of obtaining a first cluster learning model similar to the handwriting characteristics of the handwriting sample among grouped cluster learning AI models by cluster learning a plurality of handwriting data according to similar characteristics through the communication module. A method according to one embodiment may include an operation of constructing a personalized database corresponding to the user's handwriting characteristics using the handwriting sample and the first cluster learning model. A method according to one embodiment may include an operation of receiving text entered through the display. A method according to one embodiment may include an operation of converting the text into handwriting on the display based on the personalized handwriting database.

[0008] The electronic device, method, and recording medium according to various embodiments can cluster the handwriting patterns of numerous people through learning and cluster the patterns to form cluster learning AI models classified into minimum units at the phoneme level (or consonant / vowel level, character level).

[0009] Electronic devices, methods, and recording media according to various embodiments can construct a personalized database table (e.g., a handwriting stroke table) by utilizing sampling data that analyzes a personal user's handwriting samples at the phoneme level (or consonant / vowel level, character level) / minimum level and a cluster learning AI model similar to the personal user's handwriting characteristics.

[0010] The electronic device, method, and recording medium according to various embodiments can derive text input into the electronic device by referring to a personalized DB table, in units of phonemes (or consonant / vowel units, character units) or minimum units of handwriting strokes, and arrange the derived handwriting strokes according to the personal user's handwriting characteristics (e.g., character spacing, line spacing, word spacing).

[0011] Electronic devices, methods, and recording media according to various embodiments can support modification or editing of handwriting input by converting text input into phoneme units (or consonant / vowel units, character units) / minimum unit handwriting strokes.

[0012] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects may be provided, which are various effects that can be directly or indirectly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0013] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0014] FIG. 1a is a block diagram of an electronic device in a network environment according to one embodiment.

[0015] FIG. 1b illustrates the configuration of a generative AI system according to one embodiment.

[0016] FIG. 2 illustrates program configurations of an electronic device according to one embodiment.

[0017] FIG. 3 illustrates a method for converting text input into an electronic device into handwriting according to one embodiment.

[0018] FIGS. 4a and 4b illustrate screen examples for explaining a method of collecting sample handwriting according to one embodiment.

[0019] FIG. 4c illustrates examples for explaining the syllable types of Hangul according to one embodiment.

[0020] FIG. 4d illustrates an example for explaining the external character characteristics of handwriting according to one embodiment.

[0021] FIGS. 4e and FIGS. 4f illustrate an example of a personalized DB table according to one embodiment.

[0022] FIG. 5 illustrates an example of configuring a cluster learning AI model according to one embodiment.

[0023] FIGS. 6a and FIG. 6b illustrate example screens for explaining a method of converting text into handwriting according to one embodiment.

[0024] FIG. 6c illustrates an example for explaining the positioning of handwriting strokes according to one embodiment.

[0025] FIG. 7 illustrates an example for explaining various forms of handwriting strokes according to one embodiment.

[0026] FIG. 8 illustrates examples for explaining a method of requesting text to be converted into handwriting according to one embodiment.

[0027] FIG. 9 illustrates a method for converting text input into handwriting in an electronic device according to one embodiment.

[0028] FIG. 10 illustrates examples of emoticons and handwritten English characters according to one embodiment.

[0029] FIG. 11 illustrates examples of changing the attributes of handwriting according to one embodiment.

[0030] Each of the embodiments described with reference to the drawings of the present disclosure may be configured independently as a single embodiment. Each of the embodiments described with reference to the drawings of the present disclosure may operate independently as a single embodiment. At least two of the embodiments described with reference to the drawings of the present disclosure may be configured by combining. At least two of the embodiments described with reference to the drawings of the present disclosure may operate by combining. For example, at least a part of the embodiment of FIG. 1 and at least a part of the embodiment of FIG. 2 may operate by combining with each other.

[0031] When at least two of the embodiments described with reference to the drawings of the present disclosure are combined, at least some of the configurations and / or at least some operations included in each embodiment may be omitted.

[0032] The electronic device disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0033] FIG. 1a is a block diagram of an electronic device in a network environment according to one embodiment.

[0034] Referring to FIG. 1a, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160) (or display), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0035] The processor (120) includes at least one processing circuitry, and the at least one processing circuitry can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0036] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence is performed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0037] Memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input or output data for related instructions. Memory (130) may include volatile memory (132) or non-volatile memory (134). Memory (130) can store instructions executable by the processor (120) or the electronic device (101).

[0038] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0039] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0040] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0041] A display module (160) (or a display) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0042] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0043] The sensor module (176) may include at least one sensor. The sensor module (176) may detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0044] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0045] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0046] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0047] The camera module (180) includes at least one camera and can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0048] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0049] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0050] A communication module (190) may include at least one communication circuit and may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of a processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0051] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0052] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0053] According to one embodiment, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0054] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0055] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0056] An electronic device (101) according to one embodiment may support artificial intelligence (AI) or generative artificial intelligence (GAI) functions. Generative AI functions may refer to a technology that generates new content based on content and can generate a new form of AI content (or generative content) by utilizing given input data or information. Here, AI content may refer to content that is entirely generated (or reconstructed / edited) or partially generated (or reconstructed / edited) based on generative AI (e.g., 3D objects, images, videos, audio, screen information, or text).

[0057] An electronic device (101) according to one embodiment may support generative AI functions in conjunction with a server (108). At least one of the electronic device (101) or the server (108) according to one embodiment may include the configuration of the generative AI system illustrated in FIG. 1b.

[0058] FIG. 1b illustrates the configuration of a generative AI system according to one embodiment.

[0059] Referring to FIG. 1b, the generative AI system may include a user interface (10100), an AI framework (10200), a generative AI model (10300), application and service components (10400), and a database component (10500).

[0060] A user interface (e.g., user query / response interface) (10100) according to one embodiment may receive a user query. The user query input may be in the form of natural language, images, and videos. The user interface (10100) may transmit not only data regarding the user query input but also context information to an AI framework (10200). The context information may include various additional information at the time of user input. Additionally, the user query input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user query input may be in a non-natural language form that does not generate natural language, such as a menu selection (e.g., a creation request or a modification request). The user interface (10100) may output results of a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of actions requested by the user.

[0061] An AI framework (AI framework, 10200) according to one embodiment receives user query input and can coordinate and control each component necessary to perform the user's intent based on the user's query input. Such an AI framework (10200) may include a prompt design component (10210), an APIs / Plugins Management component (10230), and an output modification component (10250).

[0062] A user query or action entered in a user interface (10100) according to one embodiment may be transmitted to a prompt design component (10210). The prompt design component (10210) may be used to generate a prompt suitable for input into a large language model (LLM) or a large multimodal model (LMM). The prompt design component (10210) may be an AI component that uses machine learning algorithms or neural networks to develop better prompts over time. The prompt design component (10210) may generate a prompt by accessing a database component (10500) (e.g., a knowledge component) containing user preference data, a prompt library, and prompt examples, and transmit it to the large language model (LLM) or large multimodal model (LMM).

[0063] An API and plugin management component (10230) according to one embodiment can perform the role of communicating with external information when there is a request for additional information when transmitting user input as input to a generative model. The API and plugin management component (10230) establishes a channel to communicate with the outside of application and service components (10400) (e.g., AI Interface) through an API (application programming interface), thereby enabling access to various data sources. Additionally, the API and plugin management component (10230) can request an action through the API when the application or service needs to perform an action that ultimately executes a user query rather than an intermediate result. Information obtained from the outside can be transmitted as input to the generative model along with the user input.

[0064] An output modification component (10250) according to one embodiment can finely tune the output of a generative model. For example, the output modification component (10250) can verify whether the content generated through a language model (LLM) or a large-scale multimodal model (LMM) is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (10250) can determine the extent to which the output matches the result desired by the user and, if additional processing is required, proceed with that process. Furthermore, the output modification component (10250) can configure and provide hints to the user to avoid unwanted output.

[0065] A generative AI model (10300) according to one embodiment generally refers to an artificial intelligence neural network that generates new forms of data based on user input information. Models that generate images may include, typically, a generative adversarial network (GAN) and a variational auto encoder (VAE). For example, a generative AI model may be a Diffusion-based generative model using a VAE and a Transformer structure. Additionally, a language-generating model may refer to a model trained to output the statistically most appropriate output value based on input values. Among the generative AI models (10300), a language-generating model may be, for example, models such as CHAT-GPT 3 and CHAT-GPT 4. As another example, a large multimodal model (LMM) may be a model capable of recognizing various forms of data input, such as text, images, and voice, and generating new data corresponding to them.

[0066] An electronic device (101) according to one embodiment may provide an AI function (or operation / service) by utilizing a part of the AI ​​system operation / function of FIG. 1b. For example, the electronic device (101) may support a function of configuring clustered learning AI models clustered by similar characteristics by learning handwriting data of numerous people using AI functions, a function of selecting and loading a clustered learning AI model that is similar to the handwriting characteristics of the sampling data among the clustered learning AI models, and a function of configuring a personalized database table for text-to-handwriting conversion functions.

[0067] According to various embodiments, the electronic device (101) disclosed below may include at least some of the embodiments described in FIG. 1a and FIG. 1b. In the description of the electronic device (101) according to one embodiment of the present invention disclosed below, the same reference numerals are assigned to components substantially identical to those described in FIG. 1a and FIG. 1b above, and redundant descriptions of their functions may be omitted.

[0068] FIG. 2 illustrates program configurations of an electronic device according to one embodiment.

[0069] Referring to FIG. 2, a program of an electronic device (101) according to one embodiment (e.g., a software module implemented on a framework) includes, but is not limited to, a handwriting application (hereinafter, handwriting app) (210), a cluster learning AI model module (220), a sample collection module (230), a similar cluster matching module (240), and a positioning module (250). The operation of each component shown in FIG. 2 can be achieved through the interaction of the processor (120) and memory (130) of the electronic device (101).

[0070] A handwriting app (app, application) (210) is an application (e.g., note app, memo app, message app) capable of receiving text and / or handwriting input, and can provide various user interface (UI) screens related to handwriting input. An electronic device (101) can utilize handwriting data stored through the handwriting app (210) as a handwriting sample. In relation to the function of converting text into handwriting, if no handwriting sample is collected, the electronic device (101) can provide a UI (user interface, hereinafter UI) (e.g., sample UI) for collecting a handwriting sample from a user.

[0071] The cluster learning AI model (220) can implement operations / functions in conjunction with the AI ​​system of FIG. 1b. The cluster learning AI model (220) can be configured by loading at least some of the cluster learning AI models configured by clustering through learning the handwriting patterns of numerous people. The cluster learning AI model (220) can be configured with cluster learning AI models that have a similarity of greater than a preset ratio with the handwriting samples of the user of the electronic device (101). The cluster learning AI model (220) can output the input text characters as the result value of the handwriting stroke characters.

[0072] The sample collection module (230) can perform the function of collecting individual user's handwriting samples. The sample collection module (230) can collect handwriting data stored in the handwriting app (210) as handwriting samples (or sample data). If handwriting data is not stored in the electronic device (101), the sample collection module (230) can output a sample UI (user interface) that induces the input of handwriting samples through a display and collect handwriting samples entered by the user.

[0073] The sample collection module (230) analyzes individual user's handwriting samples in phoneme units (or consonant / vowel units, character units) / minimum units and can compose sampling data by phoneme unit (sampling data by consonant / vowel unit, sampling data by character unit). The sampling data may include information on handwriting strokes and handwriting characteristics (e.g., internal character characteristics and external character characteristics) by phoneme unit collected through the handwriting samples.

[0074] The similar cluster matching module (240) can analyze individual users' handwriting samples at the phoneme level (or consonant / vowel level, character level) and construct a personalized handwriting database table based on the handwriting samples and a cluster learning AI model that is similar to the individual user's handwriting characteristics among the cluster learning AI models. The similar cluster matching module (240) can select and load a cluster learning AI model that has a similarity ratio (e.g., 80% or more) above the standard with the user's handwriting characteristics by phoneme level (or consonant / vowel level, character level) among the cluster learning AI models constructed by clustering the handwriting patterns of numerous people.

[0075] The similar cluster matching module (240) can obtain handwriting strokes of uncollected characters as results by transmitting text characters of phoneme units (or consonant / vowel units, character units) that were not collected through the handwriting samples as input values ​​(e.g., ㅈ, ㅌ) to a selected cluster learning AI model. For example, the sentence obtained through the handwriting samples may be “Nice to meet you. The rose of Sharon has bloomed. Ganyadaerumeushuku”. The electronic device (101) may collect handwriting strokes corresponding to “ㅂ, ㅏ, ㄴ, ㄱ, ㅂ, ㅅ, ㅡ, .. ㅋ, ㅜ” among phoneme units (or consonant / vowel units, character units) from the handwriting samples, but characters such as ㅈ, ㅌ may not be collected. The similar cluster matching module (240) can collect handwriting strokes of “ㅈ, ㅌ” that have not been collected as handwriting samples through a cluster learning AI model.

[0076] The positioning module (250) can convert the input text into handwriting by referring to a personalized DB table during the process of converting the input text into handwriting, deriving handwriting stroke characters in phoneme units (or consonant / vowel units, character units), and rearranging each derived handwriting stroke character in phoneme units / minimum units according to the individual user's handwriting characteristics (e.g., character spacing, line spacing, word spacing) to convert it into handwriting input.

[0077] FIG. 3 illustrates a method for converting text of an electronic device into handwriting according to one embodiment. FIG. 4a and 4b illustrate examples of screens for explaining a method of collecting sample handwriting according to one embodiment, FIG. 4c illustrates examples for explaining syllable types of Hangul according to one embodiment, FIG. 4d illustrates an example for explaining external character characteristics of handwriting according to one embodiment, and FIG. 4e and 4f illustrate examples of personalized DB tables according to one embodiment. FIG. 5 illustrates an example of configuring a cluster learning AI model according to one embodiment.

[0078] In FIG. 3, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0079] Referring to FIG. 3, according to one embodiment, an electronic device (101) can determine whether there is a history of collecting a user's personal handwriting sample in operation 310 in relation to a function / service of converting text into handwriting.

[0080] According to one embodiment, the processor (120) can determine whether there is a history of collecting individual user handwriting samples based on receiving a request to convert text input into an electronic device into handwriting.

[0081] A handwriting sample may refer to handwriting data collected to analyze the handwriting characteristics of an individual user using an electronic device (101). The electronic device (101) may determine whether a history of sample collection exists during the text-to-handwriting conversion function or handwriting input setting process.

[0082] For example, the electronic device (101) can determine that there is a history of sample collection if handwriting data is recorded / stored in memory (130) in relation to the handwriting app (210), and can determine that there is no history of sample collection if there is no history of handwriting data being recorded / stored.

[0083] In operation 315, the processor (120) can determine whether additional handwriting samples need to be collected if there is a history of handwriting samples being collected (e.g., in operation 310, yes).

[0084] For example, if the processor (120) does not collect enough handwriting samples to cover the minimum amount required for data analysis, it may decide to collect additional handwriting samples.

[0085] According to one embodiment, the 310 operation may be omitted, and in the 315 operation, it may be determined whether to collect handwriting samples (or whether there is a situation where handwriting samples are needed).

[0086] According to one embodiment, the electronic device (101) may collect handwriting samples by analyzing the image using optical character recognition (OCR) when an image containing handwriting data is uploaded in relation to the collection of handwriting samples.

[0087] In operation 320, the processor (120) can select a method for collecting handwriting samples if there is no history of collecting handwriting samples (e.g., in operation 310, no) or if additional handwriting samples need to be collected (e.g., in operation 315, yes).

[0088] In operation 325, the processor (120) can collect handwriting samples by a sampling-based collection method if there is no previously stored handwriting data.

[0089] For example, the processor (120) of FIG. 4a <401> As illustrated in the figure, a sample UI window (420) that prompts for handwriting sample input can be displayed on the handwriting app UI screen (410). The sample UI window (420) may include a setting item (421) for changing the sample text, a closing item (422) for closing the sample text window, and a sample text (423). The user can check the sample text and input the same handwriting as the sample text on the handwriting app UI screen (410). The handwriting app UI screen (410) may include menu items (415) related to handwriting app functions (e.g., save item, pen type selection item, pen thickness selection item, pen color selection item, shape insertion item, etc.), and the menu items (415) are merely examples and may be changed or replaced with other menu items depending on the manufacturer or type of handwriting app. The processor (120) may call a UI (user interface) to change the settings of handwriting-related functions based on the selection input of the setting item (421). The processor (120) can terminate the display of the sample UI window (420) based on the selection input of the exit item (422). The processor (120) can display a UI for the corresponding handwriting app function based on the input selecting one of the menu items (415), and can perform a function for the text entered based on the user's input through the displayed UI.

[0090] The processor (120) can collect a sample of handwriting (430) identical to the sample text through the handwriting app UI screen (410).

[0091] Although not shown in the drawing, the electronic device (101) may compare the handwriting (430) input by the user through the handwriting app UI screen with the sample text (423) to visually display the progress while the user's handwriting input is being received (e.g., displaying a progress bar object or a ratio information object such as “90% input in progress”).

[0092] In operation 330, the processor (120) can collect the previously stored handwriting as sampling data when the previously stored handwriting is recorded / stored in memory (130).

[0093] For example, when handwriting is stored in connection with a handwriting app as shown in <402> of FIG. 4b, the processor (120) can load the handwriting data stored in the handwriting app UI screen (410) and use it as a handwriting sample. For example, when a handwriting memo (435) is stored that reads, “Machine learning automatically learns through experience without explicitly programming...”, the electronic device (101) can collect the handwriting recorded in the stored memo as a sample.

[0094] According to one embodiment, the electronic device (101) can update the handwriting data as sampling data whenever handwriting data (e.g., notes, memos, messages, etc.) is generated, stored, or received (e.g., received from an external electronic device) through the electronic device (101) or at specified intervals, after determining whether the handwriting data is a user's handwriting pattern. According to some embodiments, the electronic device (101) may perform the 325 operation or the 330 operation optionally, or may perform them in parallel.

[0095] If there is insufficient sampling data related to handwriting, additional sampling data can be obtained through the 325 operation, and sampling data can be updated based on input handwriting even after sampling data has been collected.

[0096] According to some embodiments, the electronic device (101) may perform 325 operations and 330 operations.

[0097] Character characteristics may include internal character characteristics and external character characteristics. Internal character characteristics may include the placement position, placement spacing, shape, size ratio, and thickness of handwriting strokes for phoneme characters (e.g., units such as ㄱ, ㄷ, ㅗ, ㅕ, a, b, 1, 2, 4, etc.) by syllable type (e.g., units such as 가, 다, 된, 위, 계, etc.). External character characteristics may include at least one of word / phrase spacing, character spacing, distance between characters, slant, word position / height, and line spacing.

[0098] Looking at Hangul as an example, the syllable unit of Hangul is Fig. 4c <403> As illustrated in [Image], it can be divided into six syllable types. Type 1 (440) may be a vertical consonant / vowel (or initial / medial) arrangement. Type 2 (441) may be a vertical consonant / vowel (or initial / medial) + consonant (or final) arrangement. Type 3 (442) may be a horizontal consonant / vowel (or initial / medial) arrangement. Type 4 (443) may be a horizontal consonant / vowel (or initial / medial) + consonant (or final) arrangement. Type 5 (444) may be a consonant (or initial) + vertical / horizontal vowel (or medial) arrangement. Type 6 (445) may be a consonant (or initial) + vertical / horizontal vowel (or medial) + consonant (or final) arrangement.

[0099] The processor (120) can separate characters within a handwriting sample into phoneme units (or consonant / vowel units, character units) and analyze internal and external character characteristics by syllable type to store them as sampling data (e.g., handwriting characteristics). The user's handwriting characteristics can be utilized in the process of positioning character characteristics (e.g., position, spacing, shape, ratio, etc.) by phoneme unit (or consonant / vowel unit, character unit) when converting input text into handwriting.

[0100] For example, in Type 1 “gi”, the position and size of the initial consonant “ㄱ” and the position and size of the medial vowel “ㅣ” can be stored as sampling data as internal character characteristics / elements. These internal character characteristics can be reflected in the positioning of the handwritten “ㄱ” and handwritten “l” when converting the text “gi” into the handwritten “gi”.

[0101] In the case of external character characteristics, the processor (120) of FIG. 4d <404> As illustrated in Fig. 4d, words / word phrases within a sentence can be recognized through the spacing of the handwriting (435) displayed on the handwriting app UI screen (410), and the spacing between words / word phrases can be measured and analyzed as external characteristics / elements of the characters. The dotted box (435) designating the recognized words / word phrases in Fig. 4d is an object for distinguishing and recognizing words / word phrases and may not be visible to the user. The processor (120) can analyze characteristics such as character spacing based on the number of characters and spacing between words / word phrases for each recognized word / word phrase.

[0102] Or the processor (120) <404> As illustrated in [Figure], in addition to word / phrase recognition, variations in the upper and lower position of the underline for each word / phrase can also be stored based on the placement position of the first character per line. For example, when examining the handwriting “automatically learning through experience without”, variations can be collected in the form of Table 1 as follows. Here, “않” is considered as the first character of the line, and the underline start position can be designated as “”. The height can refer to the height between the underline start position of “않” and the top line of “도”.

[0103] Automatically learns through experience without, starting position 067107 height 3025302530

[0104] of Fig. 4b <404> And by looking at [Table 1] as an example, the processor (120) can analyze the average starting position of the characters as 6.6 and the standard deviation as 4.9 as external character characteristics, and the average character height as 27 and the standard deviation as 8.9. As another example, the processor (120) <405> As illustrated, the handwriting (435) displayed on the handwriting app UI screen (410) can be analyzed as external character / element by measuring the line spacing (460) by separating them into line units. The handwriting characters can be reflected in the positioning of the line spacing of the handwriting. These handwriting characteristics can be reflected in the positioning of the spacing, position, and height of each word when converting text into handwriting.

[0105] Operations 3100 to 3130 can be performed independently or in parallel with operations 310 to 335.

[0106] According to some embodiments, 3100 to 3130 operations may be performed in the electronic device (101) in conjunction with at least some of the operations / functions of the server device.

[0107] In operation 3100, the electronic device (101) (or server device) can collect (or gather) a handwriting dataset (or dataset) per user for learning the handwriting of many people.

[0108] The electronic device (101) (or server device) can collect each user's handwriting data from user devices using a handwriting app, or collect handwriting data uploaded to a network (e.g., SNS, social network service / sites). For example, the electronic device (101) (or server device) can collect a first handwriting dataset from a first user, collect a second handwriting dataset from a second user, and collect an Nth handwriting dataset from an Nth user.

[0109] In operation 3110, the electronic device (101) (or server device) can divide the handwriting dataset collected per user into characters of the smallest unit (e.g., phoneme unit, consonant / vowel unit, character unit).

[0110] For example, in the case of Hangul, the minimum unit can be 14 consonants (e.g., ㄱ, ㄴ, ㄷ, ㄹ, ㅁ, ㅂ, ㅅ, ㅇ, ㅈ, ㅊ, ㅋ, ㅌ, ㅍ, ㅎ), 10 vowels ( ㅏ, ㅑ, ㅓ, ㅕ, ㅗ, ㅛ, ㅜ, ㅠ, ㅡ, ㅣ), 5 compound consonants (e.g., ㄲ, ㄸ, ㅃ, ㅆ, ㅉ), and 11 compound vowels (e.g., ㅐ, ㅒ, ㅔ, ㅖ, ㅘ, ㅙ, ㅚ, ㅝ, ㅞ, ㅟ, ㅢ).

[0111] In operation 3120, the electronic device (101) (or server device) can group characters separated by phoneme units (or consonant / vowel units, character units) into similar handwriting styles and perform cluster learning (e.g., clustering).

[0112] Cluster learning (e.g., clustering) may refer to an algorithm that learns input values ​​to distinguish them by similar characteristics and classifies them into large clustering groups. The electronic device (101) (or server device) may learn the smallest unit of characters (e.g., consonants / vowels, English letters, numbers, symbols or / and emoticons) according to the handwriting style for each user group through cluster learning.

[0113] According to one embodiment, cluster learning can be performed by classifying the forms according to syllable type. For example, when learning the cursive forms of “ㄱ”, the handwritten form of “ㄱ” in “가” and “기” of Type 1 may differ from the handwritten form of “ㄱ” in “고” of Type 3 or “공” of Type 4. Cluster learning may also be performed by classifying and learning the “ㄱ” in Type 1 and the “ㄱ” in Type 3.

[0114] For example, as illustrated in FIG. 5, a handwriting dataset (510) of User A and a handwriting dataset (515) of User B can be clustered. Although only Users A and B are shown in the drawing, the handwriting datasets for clustering can be expanded to N. An electronic device (101) (or server device) can cluster the handwriting based on the handwriting dataset (510) of User A and the handwriting dataset (515) of User B by separating the handwriting into phoneme units (e.g., consonants / vowels in the case of Korean, English, symbols) (520). For example, when learning “ㄱ” recorded in the data set of User A, the form of “ㄱ” in Type 1 and the form of “ㄱ” in Type 5 can be distinguished and learned.

[0115] In operation 3130, the electronic device (101) (or server device) can configure cluster learning AI models for each group of handwriting classified according to cluster learning.

[0116] For example, as illustrated in FIG. 5, cluster learning AI models (531, 532, ..., 53N) can be generated for each group of handwriting styles classified into similar handwriting styles by learning the handwriting styles of numerous people according to cluster learning. The cluster learning AI models (531, 532, ..., 53N) may include a first cluster learning AI model (531), a second cluster learning AI model (532), and an Nth cluster learning AI model. The minimum unit of a character in the cluster learning AI model may vary depending on the form of the language. In the case of Hangul, the minimum unit may be a phoneme unit (or consonant / vowel unit, character unit), and in the case of English, the minimum unit may be an alphabet of lowercase / uppercase letters.

[0117] According to one embodiment, a cluster learning AI model may include N data groups instead of one data group. For example, if the first cluster learning AI model (531) has multiple handwriting forms for “ㄱ”, the handwriting stroke corresponding to “ㄱ” may be output as a first form “ㄱ” stroke or a second form “ㄱ” stroke.

[0118] In operation 340, the processor (120) can load a cluster learning AI model (or a first cluster learning model) that has a similarity ratio to the handwriting characteristics of the user's sampling data among the cluster learning AI models.

[0119] For example, the cluster learning AI model can be loaded via a network in relation to the text-to-handwriting conversion function / service, or loaded from memory (130) stored by preset.

[0120] In operation 345, the processor (120) can configure a personalized database table for each phoneme unit (or consonant / vowel unit, character unit) with the user's handwriting characteristics based on sampling data and a similarity-based cluster learning AI model.

[0121] A personalized database table can be configured by listing the handwritten strokes of each character into groups 1 through N when one or more handwritten strokes are collected for each character (e.g., phoneme / minimum unit character).

[0122] For example, Fig. 4e <406> As illustrated in the figure, a sampling data table (470) of handwriting strokes by phoneme unit (or consonant / vowel unit, character unit) can be collected through handwriting samples collected from an electronic device (101). At this time, it can be confirmed that among the characters included in the sampling table (470), “ㄱ, ㄴ, ㄷ” etc. have multiple handwriting strokes collected, whereas “ㅈ, ㅌ, ㅎ” etc. have no handwriting strokes collected.

[0123] Fig. 4e <406> As illustrated in [Image], for example, if the handwriting sample is “Hello. The Rose of Sharon has bloomed. Ganya-daereum-bushuku”, the sampling data (470) may collect multiple handwriting styles for “ㄱ, ㄴ, ㅂ, ㅅ” characters / letters, but characters / letters such as ㅈ, ㅌ may not be collected. In this case, the uncollected characters / letters can be obtained as “ㅈ, ㅌ” handwriting styles through a loaded cluster learning AI model.

[0124] The processor (120) obtains a sampling data table (470) from a handwriting sample and a loaded cluster learning AI model of FIG. 4f <407> As illustrated in [Image], a personalized DB table (480) can be configured. <407> This shows a portion of the phoneme units (or consonant / vowel units, character units) of the DB table (480) illustrated in Fig. 4e, and handwritten strokes of phoneme units (or consonant / vowel units, character units) that are not illustrated may be added. The characters of each phoneme unit (or consonant / vowel unit, character unit) included in the personalized DB table (480) are not a single handwritten stroke, as in the sampling data table (470), but are of Fig. 4e <406> As illustrated in [Image], multiple handwriting strokes (e.g., Group 1 to Group N for “ㄱ”) can be listed.

[0125] According to some embodiments, the processor (120) has, in addition to the Korean DB table, a personal <408> The personalized English DB table (481) and <409> At least one of the symbol DB tables (482) shown in the drawing may be configured. In the drawing, the English DB table (481) shows uppercase forms, but may further include a lowercase DB table showing lowercase forms in addition to uppercase forms. The symbol DB table (482) is an example and may further include symbols or emoticon forms that are supported within the ASCII code.

[0126] In operation 350, the processor (120) can receive text input when there is no need to add handwriting sampling collection as in operation 315 when executing the handwriting conversion function (e.g., in operation 315, no).

[0127] For example, the processor (120) can receive text input through the handwriting app UI screen (410).

[0128] In operation 355, the processor (120) can derive handwriting strokes for the input text in phoneme units (or consonant / vowel units, character units) from a personalized DB table based on receiving a request for handwriting conversion for text.

[0129] According to one embodiment, the electronic device (101) can execute a text-to-handwriting conversion function or enter a text-to-handwriting conversion mode using various triggering methods. Triggering methods will be described by example in FIG. 8.

[0130] In 360 operation, the processor (120) can perform position positioning (or placement, rearrangement, arrangement) of the handwriting strokes derived by phoneme unit (or consonant / vowel unit, character unit).

[0131] For example, the processor (120) can perform positioning for at least one of the initial consonant (consonant) / medial vowel (vowel) / final consonant (final consonant) position, size adjustment, word spacing, character spacing, or / and line spacing for each character according to the user's handwriting characteristics (e.g., internal character characteristics and external character characteristics) analyzed from the sampling data.

[0132] In 365 operation, the processor (120) can render the handwriting result by position positioning and display the handwriting on the handwriting app UI screen.

[0133] FIGS. 6a and 6b illustrate example screens for explaining a method of converting text into handwriting according to one embodiment, and FIG. 6c illustrates an example for explaining the positioning of handwriting strokes according to one embodiment.

[0134] Referring to FIGS. 6a to 6c, according to one embodiment, an electronic device (101) is <601> As illustrated in the drawing, a handwriting app can be launched to display a handwriting app UI screen (410) on the display. The handwriting app UI screen (410) includes menu items (415) that support handwriting app functions (e.g., save item, pen type selection item, pen thickness selection item, pen color selection item, shape insertion item, etc.), but this is merely an example, and some of the menu items shown in the drawing may be omitted or replaced with other menu items.

[0135] The user can input text sentences and emoticons (610) using a Korean keyboard on the handwriting app UI screen (410). The user can request a handwriting conversion for “Nice to meet you (610).”

[0136] The electronic device (101) can divide the text “Nice to meet you” into phoneme units (or consonant / vowel units, character units) of letters (e.g., ba a n ga ba b s u b n i da a).

[0137] The electronic device (101) is based on a personalized DB table <602> As shown in [Image], handwriting strokes (e.g., ba a n ga a b s u b n i da a (620)) can be derived by phoneme unit (or consonant / vowel unit, character unit).

[0138] According to some embodiments, handwriting strokes may be listed into groups 1 through N for each character. For example, in “Ban-gap-seumnida,” the “ㅂ” may all be derived as handwriting strokes of the same typeface, but <603> As shown in the figure, the 'b' (621) of "ban," the 'b' (622) of "gap," and the 'b' (623) of "seup" may each be printed with different types of handwritten strokes.

[0139] The electronic device (101) can perform positional positioning for the handwritten strokes of the derived characters. For example, the electronic device (101) performs positional positioning for “ㅂ, ㅏ, ㄴ, ㄱ, ㅏ, ㅂ, ㅅ, ㅡ, ㅂ, ㄴ, ㅣ, ㄷ, ㅏ) by reflecting internal character characteristics and external character characteristics according to syllable type. <604> As shown in [Image], it can be converted into and displayed as "Nice to meet you" (630) in handwriting.

[0140] According to one embodiment, the electronic device (101) may arrange the size ratio and / or position differently for the initial, medial, and tonal types according to each syllable type when positioning characters as illustrated in <6c>. For example, in the case of the initial consonant “ㄱ” in “감” corresponding to Type 3, depending on the user’s handwriting characteristics The 660 form shown in, The 661 form depicted in, or <c>It can be represented in the form 662 shown in. The neutral "a" also <d>The form of 670 shown in or <e>It can be displayed in the form of 671 as shown in. In the case of the final consonant “ㅁ”, <g>Form 680 or shown in <h>The position may change, as in the form of 681.

[0141] FIG. 7 illustrates an example for explaining various forms of handwriting strokes according to one embodiment.

[0142] Referring to FIG. 7, an electronic device (101) according to one embodiment can utilize a plurality of handwriting strokes for phoneme-unit strokes.

[0143] For example, an electronic device (101) <701> As illustrated, the text "Nice to meet you" entered on the handwriting app screen (410) can be displayed by deriving handwriting strokes into characters at the phoneme level (or consonant / vowel level, character level) based on a personalized DB table and converting them into handwriting (710). At this time, the handwriting strokes corresponding to "ㅂ" can be listed as multiple N groups rather than a single form. The electronic device (101) <701> It can be output as "ㅂ" in the form of the first handwritten stroke shown in, and by probabilistic selection (e.g., random) <702> It can be output as a "ㅂ" in the form of a second handwritten stroke.

[0144] An electronic device (101) according to one embodiment may support a function to modify / edit handwriting displayed on a handwriting app screen (410) by phoneme unit (or consonant / vowel unit, character unit). For example, the electronic device (101) <701> As shown above, when touching the "ㅂ" in "반" while converted to the handwriting "반갑습니다", change the position or size of the "ㅂ", or the handwriting stroke of the "ㅂ" <702> It can be changed to the stroke of the letter "ㅂ" shown in the drawing. Although not shown in the drawing, the electronic device (101) may provide a user interface that supports editing or modifying the handwriting into characters in phoneme units (or consonant / vowel units, character units).

[0145] FIG. 8 illustrates examples for explaining a method of requesting text to be converted into handwriting according to one embodiment.

[0146] Referring to FIG. 8, an electronic device (101) according to one embodiment can receive a handwriting conversion request or enter a handwriting conversion mode in various triggering ways.

[0147] According to one embodiment, the electronic device (101) may provide a handwriting conversion mode function within a handwriting app. For example, the electronic device (101) may enter a handwriting conversion mode based on an input selecting a handwriting conversion mode menu after launching the handwriting app. The electronic device (101) <801> As illustrated in [Image], text input (810) can be received in handwriting conversion mode. The electronic device (101) can display a handwriting conversion window (815) based on the reception of text input (810) in handwriting conversion mode, and can display handwriting (817) corresponding to the text (810) entered in the handwriting conversion window (815).

[0148] According to another embodiment, the electronic device (101) receives text input within the handwriting app UI screen (410) and, based on user input that long-presses the entered text (810), <802> As illustrated in the figure, a text-related function menu window (825) may be displayed. The user may select a handwriting generation item (827) included in the text-related function menu window (825). The electronic device (101) may display a handwriting conversion window (815) based on user input selecting a handwriting generation item (827), and may display a handwriting (817) corresponding to the text (810) entered in the handwriting conversion window (815).

[0149] According to another embodiment, the electronic device (101) <803> As illustrated in the illustration, a handwriting conversion menu (817) may be provided within the menu items (415) included in the handwriting app UI screen (410). The electronic device (101) may display a handwriting conversion window (815) based on an input selecting the handwriting conversion menu (817) after inputting text (815), and may display handwriting (817) corresponding to the text in the handwriting conversion window (815).

[0150] According to one embodiment, without a handwriting conversion window (815), text may disappear from the area where text is placed and converted to handwriting and displayed.

[0151] According to one embodiment, the electronic device (101) performs operations to convert text entered in an input requesting text conversion into handwriting, and may display the converted handwriting at the time when position positioning by phoneme unit (or consonant / vowel unit, character unit) is completed (e.g., 2 to 3 seconds).

[0152] FIG. 9 illustrates a method for converting text input into handwriting in an electronic device according to one embodiment, and FIG. 10 illustrates examples of handwriting of emoticons and English characters according to one embodiment. In FIG. 9, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0153] Referring to FIGS. 9 and FIGS. 10, in relation to a function for converting text into handwriting according to one embodiment, the processor (120) of the electronic device (101) can determine the character type of the input text in operation 910.

[0154] In operation 915, if the character is a Hangul character, the processor (120) can divide the text into phoneme units (or consonant / vowel units, character units) and arrange the derived handwriting strokes into phoneme units (or consonant / vowel units, character units). For example, as illustrated in FIGS. 6a and 6b, the electronic device (101) can divide the text into phoneme units (or consonant / vowel units, character units), derive handwriting stroke characters for each phoneme unit (or consonant / vowel unit, character unit), and arrange (or position) the handwriting stroke characters by reflecting the user's handwriting characteristics.

[0155] In operation 920, the processor (120) can determine the emoticon style if the characters of the input text are emoticons.

[0156] In operation 930, the processor (120) can separate the graphic interior of the emoticon into sub-elements (e.g., eyes, nose, mouth) if the input emoticon style is a face shape. In operation 931, the processor (120) can adjust the size of each sub-element, and in operation 932, the processor can arrange each sub-element to convert the face shape emoticon into a handwriting form and display it.

[0157] For example, of Fig. 10 <1001> As shown in [Image], if the handwritten emoticon is a face shape, the sub-elements corresponding to the eyes, nose, and mouth can be separated and the size of each of the eyes, nose, and mouth can be adjusted to be expressed in a 1010 style or a 1015 style.

[0158] According to one embodiment, handwritten emoticons can be converted into different emoticon styles depending on the handwriting pattern or text type (e.g., Gothic or Gulim). For example, the emoticon style when converting Gothic type text may be different from the emoticon style when converting Gulim type text.

[0159] In operation 940, the processor (120) can derive feature points of the emoticon when the emoticon style is composite. In operation 941, the processor (120) can generate a handwriting stroke trajectory according to the feature points. In operation 942, the processor (120) adjusts the size of the handwriting stroke trajectory, and in operation 943, the processor (120) places the handwriting stroke in the center to convert and display the composite emoticon in a handwriting form.

[0160] For example, of Fig. 10 <1003> As shown in [figure], when the emoticon is a composite type, a stroke trajectory is generated according to the feature points of the emoticon, and then the size is adjusted after centering to be expressed in 1030 style or 1035 style.

[0161] In operation 950, the processor (120) can adjust the size of the handwriting stroke if the emoticon style is simple or if the input text character corresponds to Latin characters, English, or symbols, and in operation 951, the processor (120) can center the handwriting stroke to convert and display the simple emoticon, Latin characters, English, or symbols into a handwriting form.

[0162] For example, of Fig. 10 <1002> As illustrated in Fig. 10, if the emoticon corresponds to a simple type such as a heart, the handwriting stroke can be centered and then adjusted in size to be expressed in 1020 style or 1025 style. As another example, Fig. 10 <1004> As shown in [Image], when the character corresponds to English, without distinguishing sub-elements such as initial, medial, and final consonants, the handwriting stroke is placed in the center and then the size is adjusted to be expressed as 1040 style or 1045 style.

[0163] FIG. 11 illustrates examples of changing the attributes of handwriting according to one embodiment.

[0164] Referring to FIG. 11, an electronic device (101) according to one embodiment may provide a function to convert text into handwriting and then apply text attribute changes to the handwriting form.

[0165] for example, <1101> The electronic device (101) can convert the text "Hello" into a handwriting style such as 1010 through the handwriting app (210). When a request to change the text color is made through the function menu of the handwriting app, the electronic device (101) can change and apply the color of the handwriting as 1111. The electronic device (101) can also apply an underline attribute as 1112 or a strikethrough attribute as 1113. As another example, the electronic device (101) can apply the font attribute of the text to the handwriting. The electronic device (101) can change the size of the handwriting in proportion to the size of the input text. Depending on the text attribute of the Gothic font, the electronic device (101) can change the font attribute of the handwriting to a bold attribute as 1114 or to an italic attribute as 1115.

[0166] For example, when changing to italic properties, since italic text has a slanted structure, when arranging initial, medial, and final consonants, <1102> As with 1120, each placement area can be represented by changing it into a tilted shape. Although not shown in the drawing, the electronic device (101) <1103> In cases where it is expressed as a bold attribute, such as 1130, a user interface can be provided to adjust the thickness by changing the bold thickness. In this case, since the handwriting becomes illegible when the thickness of the strokes changes to a thicker level, if the bold thickness changes to a thickness exceeding the standard... <1104> It can be expressed by creating a gap between the initial, medial, and final consonants, as in 1140.

[0167] An electronic device according to one embodiment may include a communication module comprising at least one communication circuit. An electronic device according to one embodiment may include a display. An electronic device according to one embodiment may include a memory for storing instructions. An electronic device according to one embodiment may include a processor (120). When the instructions according to one embodiment are executed by the processor, the electronic device may collect a user's handwriting sample entered in handwriting based on at least one of the display and the memory. The instructions according to one embodiment may identify the handwriting characteristics of the collected handwriting sample and, through the communication module, cluster learning a plurality of handwriting data according to similar characteristics to obtain a first cluster learning model among the grouped cluster learning AI models that is similar to the handwriting characteristics of the handwriting sample. The instructions according to one embodiment may construct a personalized database corresponding to the user's handwriting characteristics using the handwriting sample and the first cluster learning model. The instructions according to one embodiment may receive text input through the display. The instructions according to one embodiment may convert the text into handwriting on the display based on a personalized handwriting database.

[0168] Instructions according to one embodiment may enable the electronic device to convert each character included in the text into handwriting by classifying it into phoneme units, consonant / vowel units, character units, or minimum unit forms.

[0169] Instructions according to one embodiment may enable the electronic device to classify the syllable types of each character included in the handwriting sample when the handwriting is a Hangul character, and to analyze the handwriting attributes of each phoneme unit, consonant / vowel unit, and character unit for each syllable type to construct sampling data for each phoneme unit.

[0170] Instructions according to one embodiment may enable the electronic device to classify the characters included in the handwriting sample into minimum units when the handwriting is English, a symbol, or a Latin character, and to analyze the character attributes for each minimum unit to construct sampling data for each minimum unit.

[0171] According to one embodiment, the character attributes include internal character characteristics and external character characteristics, the internal character characteristics include the placement position, size ratio, and thickness of the handwriting stroke of the minimum unit or the phoneme unit, and the external character characteristics may include word spacing, word-specific position height, and line spacing.

[0172] Instructions according to one embodiment may enable the electronic device to receive a result of a handwriting stroke by transmitting a text character of a phoneme unit or a minimum unit that was not collected from the handwriting sample as an input value to the first cluster learning AI model, and to construct a personalized database composed of phoneme units or minimum units based on the handwriting strokes received from the first cluster learning AI model and the handwriting strokes obtained from the handwriting sample.

[0173] Instructions according to one embodiment allow the electronic device to use the stored handwriting data as the handwriting sample when handwriting data is stored in the memory, and when handwriting data is not stored in the memory, to display a user interface guiding the input of a handwriting sample on the display and to collect the handwriting input through the display as the handwriting sample.

[0174] Instructions according to one embodiment may enable the electronic device to adjust the position or size of the handwriting strokes corresponding to each character for position positioning of the handwriting strokes obtained by dividing the input text into phoneme units or minimum units.

[0175] A personalized database according to one embodiment is composed of handwriting strokes corresponding to characters at the minimum unit or phoneme unit for each language, and may include at least one or N handwriting stroke data for each character.

[0176] A method for converting text of an electronic device into handwriting according to one embodiment may include an operation of collecting a user's handwriting sample entered as handwriting based on at least one of a display and a memory. A method according to one embodiment may include an operation of verifying the handwriting characteristics of the collected handwriting sample. A method according to one embodiment may include an operation of obtaining a first cluster learning model similar to the handwriting characteristics of the handwriting sample among grouped cluster learning AI models by cluster learning a plurality of handwriting data according to similar characteristics through the communication module. A method according to one embodiment may include an operation of constructing a personalized database corresponding to the user's handwriting characteristics using the handwriting sample and the first cluster learning model. A method according to one embodiment may include an operation of receiving text entered through the display. A method according to one embodiment may include an operation of converting the text into handwriting on the display based on the personalized handwriting database.

[0177] The embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" each may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0178] The term “module” as used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0179] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0180] According to one embodiment, the method according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0181] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components of the multiple components in the same or similar manner as those performed by the corresponding components among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.< / h> < / g> < / e> < / d> < / c>

Claims

1. In an electronic device (101), A communication module (190) including at least one communication circuit Display (160), Memory (130) for storing instructions; and The electronic device includes a processor (120), and when the instructions are executed by the processor, the electronic device Collecting a user's handwriting sample entered in handwriting based on at least one of the above display and the above memory, and Confirm the handwriting characteristics of the above-collected handwriting samples, and A first cluster learning model similar to the handwriting characteristics of the handwriting sample is obtained among the grouped cluster learning AI models by cluster learning multiple handwriting data based on similar characteristics through the above communication module, and Using the above handwriting sample and the above first cluster learning model, a personalized database corresponding to the user's handwriting characteristics is constructed, and Receive text entered through the above display, and An electronic device that converts the above text into handwriting on the display based on a personalized handwriting database.

2. In Paragraph 1, The above instructions are for the electronic device, An electronic device that converts each character included in the above text into handwriting by classifying it into phoneme units, consonant / vowel units, character units, or minimum unit forms.

3. In Paragraph 1, The above instructions are for the electronic device, If the above handwriting consists of Hangul characters, classify the syllable types of each character included in the above handwriting sample, and An electronic device that analyzes the handwriting attributes of each phoneme unit, consonant / vowel unit, or character unit according to the above-mentioned syllable type to construct sampling data by phoneme unit.

4. In Paragraph 1, The above instructions are for the electronic device, An electronic device that, when the handwriting is English, a symbol, or a Latin character, classifies the characters included in the handwriting sample into minimum units, analyzes the character attributes for each minimum unit, and constructs sampling data for each minimum unit.

5. In Paragraph 3 or 4, The above character attributes include internal character characteristics and external character characteristics, and The internal characteristics of the above characters include the placement position, size ratio, and thickness of the handwriting strokes of the minimum unit or the phoneme unit, and The above-mentioned external character characteristics include word spacing, word-by-word position height, and line spacing in an electronic device.

6. In Paragraph 5, The above instructions are for the electronic device, Text characters in phoneme units or minimum units that were not collected from the above handwriting samples are transmitted as input values ​​to the above first cluster learning AI model to receive result values ​​of handwriting strokes, and An electronic device that configures a personalized database composed of phoneme units or minimum units based on handwriting strokes received from the first cluster learning AI model and handwriting strokes obtained from the handwriting sample.

7. In Paragraph 5, The above instructions are for the electronic device, If handwriting data is stored in the above memory, the stored handwriting data is used as the above handwriting sample, and An electronic device that, when handwriting data is not stored in the memory, displays a user interface guiding the input of a handwriting sample on the display and collects the handwriting input through the display as the handwriting sample.

8. In Paragraph 5, The above instructions are for the electronic device, An electronic device that divides the input text into phoneme units or minimum units to obtain handwriting strokes, and adjusts the position or size of the handwriting strokes corresponding to each character for positional positioning.

9. In Paragraph 5, The above personalized database consists of handwriting strokes corresponding to characters at the smallest unit or phoneme unit level for each language, and An electronic device comprising at least one or N handwriting stroke data for each character.

10. A method for converting text on an electronic device into handwriting, The operation of collecting a user's handwriting sample entered by hand based on at least one of a display and memory; An operation to verify the handwriting characteristics of the above-mentioned collected handwriting samples; The operation of obtaining a first cluster learning model similar to the handwriting characteristics of the handwriting sample among grouped cluster learning AI models by cluster learning multiple handwriting data according to similar characteristics through a communication module; The operation of constructing a personalized database corresponding to the user's handwriting characteristics using the above handwriting sample and the above first cluster learning model; The operation of receiving text input through the above display; A method comprising the operation of converting the above text into handwriting on the display based on a personalized handwriting database.

11. In Paragraph 10, The above operation of converting to handwriting is, A method characterized by converting each character included in the above text into handwriting by classifying it into phoneme units, consonant / vowel units, character units, or minimum unit forms.

12. In Paragraph 10, The operation of collecting the above handwriting samples is, A method further comprising, when the above handwriting is Hangul characters, classifying the syllable types of each character included in the above handwriting sample, and analyzing the handwriting attributes of each phoneme unit, consonant / vowel unit, and character unit for each said syllable type to construct phoneme-unit sampling data.

13. In Paragraph 10, The operation of collecting the above handwriting samples is, A method further comprising, when the handwriting is English, a symbol, or a Latin character, classifying the characters included in the handwriting sample into minimum units and analyzing the character attributes for each minimum unit to construct sampling data for each minimum unit.

14. In Paragraph 12 or 13, The above character attributes include internal character characteristics and external character characteristics, and The internal characteristics of the above characters include the placement position, size ratio, and thickness of the handwriting strokes of the minimum unit or the phoneme unit, and The above-mentioned external character characteristics include word spacing, word-specific position height, and line spacing.

15. In Paragraph 10, The operation of configuring the above personalized database is, A method characterized by transmitting text characters in phoneme units or minimum units that were not collected from the handwriting samples as input values ​​to the first cluster learning AI model to receive result values ​​of handwriting strokes, and constructing a personalized database composed of phoneme units or minimum units based on the handwriting strokes received from the first cluster learning AI model and the handwriting strokes obtained from the handwriting samples.

Images