Electronic device, method, and non-transitory computer-readable storage medium for grouping text of electronic document

Abstract

An electronic device according to an embodiment may receive a first input for executing a file including a first object including a first text, a second object including a second text, and a third object positioned between the first text and the second text and including a third text. The electronic device may display the first text, the second text, and the third text on the basis of the first input. The electronic device may, on the basis of a second input received for the first text and the second text, adjust the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text and display, in the electronic document, a visual object indicating that the first text, the third text, and the second text are selected.

Description

Electronic device, method, and non-transitory computer-readable storage medium for grouping text in an electronic document

[0001] The present disclosure relates to an electronic device, a method, and a non-transitory computer-readable storage medium for grouping text in an electronic document (e.g., an electronic document based on portable document format (PDF) and / or a file including the electronic document).

[0002] An electronic document is described as a document in electronic form (e.g., a file and / or a bitstream transmitted over a network). For example, an electronic document based on text, graphics, images, fonts, or a combination thereof may be stored within a file based on a standard related to the portable document format (PDF) (e.g., the International Standard Organization (ISO) 32000). An electronic device running a software application for electronic documents (e.g., an electronic document viewer) can identify text, graphics, and / or images, each occupying a portion of the electronic document, from the file. The electronic device can display the text, graphics, and / or images on a display having a layout represented by the electronic document.

[0003] The above information may be provided as background information to aid in understanding the present disclosure. None of the above is claimed to be prior art related to the present disclosure or can be used in making decisions related to prior art.

[0004] According to an embodiment, an electronic device may include at least one circuit comprising a display, a memory including one or more storage media and storing instructions, and a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a first input to execute a file comprising a first object including a first text, a second object including a second text, and a third object including a third text positioned between the first text and the second text. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the first text, the second text, and the third text based on the first input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a visual object within the electronic document indicating that the first text, the third text, and the second text are selected by adjusting the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text, based on a second input received with respect to the first text and the second text.

[0005] In one embodiment, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions, when executed by an electronic device including a display, may cause the electronic device to receive a first input to execute a file including a first object including first text, a second object including second text, and a third object including third text positioned between the first text and the second text. The instructions, when executed by the electronic device, may cause the electronic device to display an electronic document including the first text, the second text, and the third text based on the first input. The instructions, when executed by the electronic device, may cause the electronic device to display, within the electronic document, a visual object indicating that the first text, the third text, and the second text are selected by adjusting the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text, based on a second input received with respect to the first text and the second text.

[0006] In one embodiment, a method of an electronic device including a display may be provided. The method may include receiving a first input for executing a file representing an electronic document. The method may include identifying objects within the file, which are groups of characters included in the electronic document, based on the first input, and displaying the electronic document on the display. The method may include determining a plurality of groups by grouping at least two characters positioned adjacent to each other within the electronic document using positional information of the characters represented by the objects. The method may include determining an order of the plurality of groups based on an arrangement of the plurality of groups within the electronic document. The method may include determining one or more characters using the order determined for the plurality of groups based on receiving a second input for selecting one or more characters positioned between a first character and a second character among the characters.

[0007] In one embodiment, an electronic device may include a display, a memory including one or more storage media and storing instructions, and at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a first input for executing a file representing an electronic document. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify objects within the file, groups of characters included in the electronic document, based on the first input, and to display the electronic document on the display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine a plurality of groups by grouping at least two characters positioned adjacent to each other within the electronic document using positional information of the characters represented by the objects. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine an order of the plurality of groups based on an arrangement of the plurality of groups within the electronic document. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine one or more characters using the determined order for the plurality of groups based on receiving a second input for selecting one or more characters positioned between a first character and a second character among the characters.

[0008] Figure 1 illustrates an exemplary operation of an electronic device for displaying an electronic document.

[0009] FIG. 2 illustrates a schematic block diagram of an electronic device according to one embodiment.

[0010] Figure 3 illustrates a flowchart of the operation of an electronic device for displaying an electronic document.

[0011] FIG. 4 illustrates an exemplary operation of an electronic device for segmenting text represented by objects in a file representing an electronic document.

[0012] FIG. 5 illustrates an exemplary operation of an electronic device for determining groups of text based on spacing between the texts.

[0013] FIG. 6 illustrates an exemplary operation of an electronic device for identifying groups of texts based on spacing between characters.

[0014] Figure 7 illustrates an exemplary operation of an electronic device based on spacing (or distances) between groups of text.

[0015] FIG. 8 illustrates an exemplary operation of an electronic device for grouping line groups to identify one or more paragraph groups.

[0016] Figure 9 illustrates the operation of an electronic device for determining the order of groups of text included in an electronic document.

[0017] Figure 10 illustrates an exemplary operation of an electronic device displaying an electronic document.

[0018] FIG. 11 is a block diagram of an electronic device within a network environment according to various embodiments.

[0019] Hereinafter, various embodiments of this document are described with reference to the attached drawings.

[0020] The various embodiments of this document and the terminology used therein are not intended to limit the technology described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, and / or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and / or B", "A, B, or C", or "at least one of A, B, and / or C" may include all possible combinations of the items listed together. Expressions such as "first", "second", "first", or "second" may modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a component (e.g., a first component) is “(functionally or communicatively) connected” or “connected” to another component (e.g., a second component), said component may be directly connected to said other component, or may be connected via another component (e.g., a third component).

[0021] FIG. 1 illustrates an exemplary operation of an electronic device (101) displaying an electronic document (130). The electronic device (101) can control a display (110) to display at least a portion of the electronic document (130). An exemplary hardware configuration included in the electronic device (101) is described with reference to FIG. 2.

[0022] Referring to FIG. 1, an exemplary state of an electronic device (101) displaying an electronic document (130) is illustrated. The electronic device (101) may display at least a portion of the electronic document (130) on at least a portion of the display (110). For example, the electronic device (101) may display at least a portion of the electronic document (130) on the display (110) in response to an input for opening, executing, or viewing the electronic document (130) and / or a file representing the electronic document (130) (e.g., file (280) of FIG. 2). The electronic device (101) may display at least a portion of the electronic document (130) by executing a software application for displaying the electronic document (130) (hereinafter, referred to as an electronic document viewer and / or viewer application). For example, the electronic document (130) may be stored as a file in the memory of the electronic device (101). For example, the electronic document (130) may be stored in an external electronic device (e.g., a server) connected to the electronic device (101) via a network. The electronic device (101) may obtain a bitstream representing at least a portion of the electronic document (130) from the external electronic device. For example, the electronic document (130) may be in the form of a file stored in the electronic device (101) and / or a bitstream transmitted to the electronic device (101) via a network.

[0023] Referring to FIG. 1, the electronic device (101) may display a user interface (UI) based on a viewer application on a display (110). The electronic device (101) may display a title bar and / or a status bar on a portion (121) of a display area including the top of the display (110). The title bar may display a name of an electronic document (130) (e.g., “A_doc”). The status bar may display icons indicating the status of the electronic device (101), such as the state of charge (SOC) of the battery, the current time, and / or a WiFi connection.

[0024] Referring to FIG. 1, the electronic device (101) can display at least a portion of an electronic document (130) on a portion (122) on a display (110). Although the portion (122) is illustrated as being positioned below the portion (121), the positional relationship between the portions (121, 122) is not limited to the embodiment. The portion (122) may be referred to as a browser portion of a viewer application. Through the portion (122), the electronic device (101) can display at least a portion of the content of the electronic document (130) including the first text (131). In response to a touch gesture performed on the portion (122) (e.g., a drag gesture based on the contact of two fingers and / or a swipe gesture), the electronic device (101) can change or replace (e.g., scroll) the portion of the electronic document (130) displayed through the portion (122) with another portion. In response to another touch gesture performed on the portion (122) (e.g., a pinch-to-zoom gesture based on the contact of two fingers), the electronic device (101) may zoom in or out on the electronic document (130) displayed through the portion (122).

[0025] Referring to FIG. 1, the electronic device (101) can display executable objects, each corresponding to a function for editing an electronic document (130), on a portion (123) on the display (110). Although portion (123) is illustrated as being positioned below portion (122), the positional relationship between portions (122, 123) is not limited to the embodiment. For example, portion (123) can be overlapped on portion (122) in the form of a pop-up window. For example, portion (123) can be conditionally displayed based on user input.

[0026] For example, the executable objects represented by the portion (123) may each correspond to a function for adding text entered via a software keyboard (or a virtual keyboard, or a physical keyboard paired with the electronic device (101)) to the electronic document (130), a function for adding at least one stroke having a shape similar to a pen drawing of a ballpoint pen on the electronic document, a function for adding at least one stroke having a shape similar to a pen drawing of a liner on the electronic document (130), a function for removing one or more strokes added to the electronic document (130) (e.g., strokes representing hand-writing and / or hand-drawing), a function for selecting strokes added to the electronic document (130) (e.g., a lasso function), a function for identifying texts having a particular font from a plurality of strokes, and / or a function for at least partially undoing editing actions applied to the electronic document (130).

[0027] Referring to FIG. 1, the electronic device (101) may display executable objects for changing or replacing a screen (e.g., a screen including parts (121, 122, 123) and based on the execution of a viewer application) being displayed through the display (110) on a portion (124) on the display (110) with another screen. The executable objects displayed through the portion (124) may each correspond to a function for displaying another screen that was displayed before the screen being displayed through the display (110) within the state of FIG. 1 (e.g., a back function), a function for displaying a designated screen referred to as a launcher screen (or home screen) (e.g., a home function), and / or a function for displaying a list of one or more screens that were displayed on the display (110) (or executed by the electronic device (101).

[0028] According to one embodiment, an electronic device (101) can identify or display an electronic document (130) represented by a file based on a portable document format (PDF). Information representing the content of the electronic document (130) may be stored within the PDF-based file. The file may include objects representing elements of the content, respectively, as the information. The objects may be referred to as document objects and / or document elements. An exemplary structure of a file conforming to PDF is described with reference to FIG. 3.

[0029] Humans conventionally write documents sequentially. Furthermore, humans read or perceive documents linearly. However, PDF is not standardized so that characters contained in an electronic document are stored sequentially within the file. For example, to reduce the file size or efficiently render the electronic document represented by the file, objects contained in the file may be arranged in an order different from the direction and / or order in which a human would perceive or read the characters in the electronic document. In the example, while the electronic device (101) sequentially renders the objects contained in the file, different portions of the content corresponding to the objects may be visualized or arranged nonlinearly within the electronic document (130) according to the order.

[0030] For example, while rendering an electronic document (130) using a file, the order in which portions of the content are displayed may differ from the direction and / or order in which a human reads the electronic document. For example, in a case where first to fifth objects are sequentially stored in a file, the electronic device (101) may identify text (f1) ("electronic document") located in a portion of the electronic document (130) corresponding to the first object. Based on identifying a second object following the first object from the file, the electronic device (101) may render text (f2) ("Author: Sammy Kim") indicated by the second object in the electronic document (130). The electronic device (101) may parse a third object following the second object and render text (f3) ("Dealing with the future of electronic documents") indicated by the third object. The electronic device (101) may identify a fourth object located following the third object in the file. An electronic device (101) that has identified a fourth object can display or render an electronic document (130) that includes text (f4) corresponding to the fourth object (“This document examines various specifications of electronic documents”). An electronic device (101) that has detected a fifth object located next to the fourth object in the file can render text (f5) indicated by the fifth object (“The electronic document is computer data transmitted over the Internet”). Referring to FIG. 1 , while the first to fifth objects are parsed, the order in which texts (f1, f2, f3, f4, f5) are rendered within the electronic document (130) can be independent of (or different from) the human gaze viewing the texts (f1, f2, f3, f4, f5).

[0031] Referring to FIG. 1, an exemplary state of an electronic device (101) displaying an electronic document (130) by sequentially rendering objects included in a file is illustrated. Within a state of displaying an electronic document (130) including texts (f1, f2, f3, f4, f5) respectively corresponding to a plurality of objects, the electronic device (101) may at least partially extract content of the electronic document (130). For example, the electronic device (101) may at least partially extract text of the electronic document (130) based on a user input. The operation of extracting text may include an operation of storing the text in a memory of the electronic device (101), referred to as a clipboard, and / or a buffer. The operation of extracting text may include an operation of executing a software application different from a viewer application displaying the electronic document (130) by using the text. The act of extracting text may include distributing or transmitting the text using email, messenger, and / or short message service (SMS). The act of extracting text may include processing the text using artificial intelligence-based services, such as translation and / or summarization.

[0032] Referring to FIG. 1, in a state where at least a portion of an electronic document (130) is displayed on a display (110), an electronic device (101) may receive an input for selecting text of the electronic document (130). The input may be detected based on a drag gesture (e.g., continuous movement of a contact point on the display (110)) performed on a portion (e.g., portion (122)) of the display (110) where the electronic document (130) is displayed. Referring to FIG. 1, an electronic device (101) that detects a drag gesture from a position where a specific word (e.g., “document”) of the electronic document (130) is displayed to a position where another word (e.g., “computer”) is displayed may receive an input for selecting text between the words within the electronic document (130).

[0033] While exemplary operations for receiving the input based on a drag gesture have been described, the embodiments are not limited thereto. For example, the electronic device (101) may receive or detect an input indicating a selection of text based on a tap gesture performed on the display (110) (or a double tap gesture, and / or a long-touch gesture detected based on a finger remaining in contact with the display (110) for more than a specified period of time, such as 1.5 seconds).

[0034] Referring to FIG. 1, a user who wishes to select text in an electronic document (130) (e.g., a user who wishes to translate at least a portion of the text) may specify the text based on the order in which the text is recognized. Meanwhile, since objects representing the text in the electronic document (130) are arranged in an order different from the order described above, the order of the objects (or the order in which the text is rendered based on the objects) may result in the selection of characters that do not match the order and / or intent recognized by the user.

[0035] When processing an input for selecting text by using the order of objects in the file while displaying an exemplary electronic document (130) of FIG. 1, in response to a drag gesture from a position where "document" is displayed in the text (f1) to a position where "computer" is displayed in the text (f5), texts (f2, f3, f4) among the texts (f1, f2) can be selected. Furthermore, based on the order of the objects, text (e.g., "Document author: Sammy Kim This document covers the future of electronic documents. This document examines various standards for electronic documents, and electronic documents are computers transmitted over the Internet") can be determined to have been selected by the user. Based on the order (or concept) in which the texts are perceived by the user, the user performed the drag gesture to select a text between "document" and "computer" (e.g., "document" among "electronic document" in text (f1), and "computer that transmits electronic document over the Internet" within text (f5), rather than texts (f2, f3, f4). That is, a text different from the intention of the user who performed the drag gesture may be selected.

[0036] According to one embodiment, the electronic device (101) can determine or estimate the order in which texts of an electronic document (130) displayed on the display (110) are perceived by the user in order to identify or select a portion of the text that matches the user's intent described above. The electronic device (101) can identify the order by using the positions of the texts (f1, f2, f3, f4, f5) within the electronic document (130) (or position data and / or position information corresponding to the texts (f1, f2, f3, f4, f5)).

[0037] An exemplary operation of the electronic device (101) to identify or determine the order according to one embodiment is described with reference to FIGS. 3 to 9. For example, the electronic device (101) may identify or determine the order in which characters are to be recognized (e.g., reading order and / or recognition order) by measuring spacing between characters, spacing between lines represented by a set of characters, and / or spacing between paragraphs that are sets of lines, and / or blank areas (or margin areas, or padding areas) formed within the electronic document (130).

[0038] Referring to FIG. 1, an exemplary state is illustrated in which an electronic device (101) that has received an exemplary user input (e.g., an input indicated by a drag gesture from a location where "document" is displayed in text (f1) to a location where "computer" is displayed in text (f5)) visually highlights text in an electronic document (130) corresponding to the user input relative to the remaining text. Referring to FIG. 1, the electronic device (101) can visually highlight "document" included in the text (f1) and "computer that transmits electronic document over the Internet" included in the text (f5) relative to the remaining text. The electronic device (101) can display a visual object (141) at the starting point of the visually highlighted text (e.g., the left edge of "document" in the text (f1)).

[0039] Referring to the exemplary state of FIG. 1, the electronic device (101) may display a visual object (142) at the end point of visually emphasized text (e.g., the right edge of “computer” in the text (f5)). Between the visual objects (141, 142), a shape (or visual object, or dimming layer) having a designated color and / or designated transparency and having a shape superimposed on the text may be superimposed on the text to visually emphasize the text. When a drag gesture is further received for at least one of the visual objects (141, 142), the electronic device (101) may change the range of the visually emphasized text by using the position of at least one of the visual objects (141, 142) moved by the drag gesture. The range may be changed according to the order of the text in the electronic document (130) identified by the electronic device (101).

[0040] Within the exemplary state of FIG. 1, the electronic device (101) may display a menu (150) indicating executable functions using visually highlighted text. The menu (150) may be displayed on the display (110). The menu (150) may be displayed as an overlay on the visually highlighted text. The menu (150) may include executable objects representing each of the functions. The executable objects may be referred to as items (or visual objects) within the menu (150).

[0041] For example, the menu (150) may include at least one of an executable object mapped to a function of searching the Internet using visually highlighted text (e.g., an object having the text "Search"), an executable object mapped to a function of saving visually highlighted text, at least temporarily, in a clipboard (e.g., an object having the text "Copy"), an executable object mapped to a function of applying a visual effect (e.g., a highlight effect) to the visually highlighted text (e.g., an object having the text "Highlight"), an executable object mapped to a function of generating a summary (e.g., another text having a length less than or equal to that of the visually highlighted text) from the visually highlighted text (e.g., an object having the text "Summary"), and / or an executable object mapped to a function of translating the visually highlighted text (e.g., an object having the text "Translate"). The order of the executable objects included in the menu (150) is not limited to the embodiment of FIG. 1. According to embodiments, at least one of the executable objects included in the menu (150) may be omitted, and additional executable objects not shown in the menu (150) may be displayed.

[0042] In one embodiment, the electronic device (101) may receive an input for executing a function associated with visually highlighted text via the menu (150). For example, upon receiving an input for selecting an executable object having the text "translate," the electronic device (101) may execute a function corresponding to the input (e.g., a translation function) using the visually highlighted text.

[0043] As described above, according to one embodiment, the electronic device (101) may receive a first input for executing a file representing an electronic document (130). Based on the first input, the electronic device (101) may identify objects within the file, which represent groups of characters included in the electronic document (130). The electronic device (101) may identify the objects and display the electronic document (130) on the display (110). The electronic device (101) may determine a plurality of groups by grouping at least two characters positioned adjacent to each other within the electronic document (130) using positional information of the characters represented by the objects. Based on the arrangement of the plurality of groups within the electronic document (130), the electronic device (101) may determine an order of the plurality of groups.

[0044] For example, the electronic device (101) may identify or determine, from among the characters, the one or more characters based on receiving a second input for selecting one or more characters located between a first character (e.g., “document” in the text (f1)) and a second character (e.g., “computer” in the text (f5)), using the order. For example, if a file representing an electronic document (130) represents objects that are arranged non-linearly, independently of the recognition order of texts in the electronic document (130), the electronic device (101) may estimate the recognition order and, using the estimated recognition order, identify the one or more characters corresponding to the second input.

[0045] As described above, since the electronic device (101) can estimate the recognition order and select or extract text from the electronic document (130) using the estimated recognition order, the user experience (e.g., copying, translating, and / or summarizing) associated with the extracted text can be improved. For example, in response to an input for at least partially selecting text from the electronic document (130), the electronic device (101) can select (e.g., visually highlight over other portions) at least a portion of the text that matches the intent of the user who provided the input. Since at least a portion of the text is selected based on the recognition order, the electronic device (101) can accurately identify or select the text corresponding to the intent. The electronic device (101) can utilize at least a portion of the text from the electronic document (130) based on the recognition order to perform various functions associated with text processing, such as copying, translating, and / or summarizing.

[0046] Below, with reference to FIG. 2, an exemplary hardware configuration of the electronic device (101) of FIG. 1 is described.

[0047] FIG. 2 illustrates a schematic block diagram of an electronic device (101) according to one embodiment. The electronic device (101) of FIG. 2 may include the electronic device (101) of FIG. 1. Referring to FIG. 2, the electronic device (101) may be one of various forms of electronic devices, such as a laptop PC (personal computer) (290), smartphones (291) having various form factors (e.g., a bar-type smartphone (291-1), a foldable-type smartphone (291-2), or a sliderable (or rollable) type smartphone (291-3) having the appearance of the electronic device (101) of FIG. 1), a tablet PC (292), a head-mounted display (or head-wearable device) device (293), a watch (294), and other similar computing devices (not shown).

[0048] In one embodiment, the electronic device (101) may be referred to as a mobile device, a user equipment (UE) (or user terminal), a multi-function device, a portable communication device, a portable device, or a server. The form factor of the electronic device (101) is not limited to the exemplary form factors illustrated in FIG. 2. For example, the electronic device (101) may be included as an electronic control unit (ECU) in a vehicle (e.g., an electric vehicle (EV)). For example, the electronic device (101) may include a projector for projecting light to the outside. For example, the electronic device (101) may have any suitable form for outputting an electronic document (e.g., the electronic document 130 of FIG. 1). For example, the electronic device (101) may include a printer that outputs ink (or toner) so that the electronic document can be drawn on a surface of an object such as paper.

[0049] Referring to FIG. 2, according to one embodiment, an electronic device (101) may include a processor (210) and / or a memory (220). The electronic device (101) may further include a display (110). The processor (210) may be electrically and / or operatively coupled with the memory (220) and / or the display (110). Electrical coupling of the electronic components may include a state in which a wired signal path (or a connection for wireless communication) for transmitting signals is established between the electronic components. Operationally coupling of the electronic components may include a state in which the electronic components are directly coupled (or a state in which the electronic components are indirectly coupled) such that one of the electronic components controls another electronic component. Referring to FIG. 2, an electrical connection between the processor (210), the memory (220), and the display (110) based on electronic components, referred to as a communication bus (202), is schematically illustrated. Through a communication bus (202), the processor (210), memory (220), and display (110) can be communicatively coupled.

[0050] Referring to FIG. 2, a processor (210) of an electronic device (101) may include circuits (e.g., processing circuits and / or cores) for performing operations (e.g., arithmetic operations and / or logical operations) on data. Binary codes (e.g., instructions) representing the operations may be input to the processor (210). The processor (210) may include a central processing unit (CPU), a graphic processing unit (GPU), and / or a neural processing unit (NPU). The processor (210) may be referred to as an application processor (AP) and / or a system on a chip (SoC). The processor (210) may have a structure (e.g., a multi-core structure based on a combination of multiple core circuits such as a dual core, a quad core, a hexa core, or an octa core) for loading (or fetching) and / or executing multiple instructions simultaneously. Within an electronic device (101) comprising at least one processor, including a processor (210), the at least one processor may individually or collectively perform the operations of the present disclosure. For example, the at least one processor may individually and / or collectively perform the operations of FIG. 3 by executing instructions stored in the memory (220).

[0051] The memory (220) of FIG. 2 may include a circuit for storing data (or instructions) input to or output from the processor (210). The memory (220) may include volatile memory, such as random-access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM). The non-volatile memory may be referred to as storage. The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, solid state drive (SSD), and embedded multimedia card (eMMC). The memory (220) may include one or more storage media (e.g., the volatile memory and / or non-volatile memory described above) located in a distributed manner in the electronic device (101). The processor (210) of the electronic device (101) may execute instructions of the memory (220) within the electronic device (101) to perform functions and / or operations (e.g., the operations of FIG. 3) indicated by the instructions.

[0052] The display (110) of the electronic device (101) may include a circuit for visualizing information provided from the processor (210). The display (110) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). The embodiment is not limited thereto, and the display (110) may include electronic paper. The display area (or active area) of the display (110) may include an area where light is emitted, formed by pixels (e.g., activated pixels) of the display (110). The display (110) may include a sensor (e.g., a touch sensor) for detecting an external object (e.g., a user's finger) on the display (110). The sensor may be included in the display (110) in the form of a panel (e.g., a touch sensor panel (TSP)).

[0053] Referring to FIG. 2, programs (e.g., a viewer application (260) and / or an electronic document library (250)) are illustrated that are executed by a processor (210) to create or read a file (280) representing an electronic document. The programs may be independently installed in memory (220) (e.g., a collection of instructions and resources, referred to as a package) or may be stored in memory (220) as sub-routines of a single program (or an applet or a dynamic link library (DLL)).

[0054] For example, the processor (210) executing the viewer application (260) may display a UI (e.g., the UI illustrated in FIG. 1) of the viewer application (260), which includes at least a portion of an electronic document represented by a file (280), on the display (110). Through the UI, the processor (210) may provide the user with functions related to the text of the electronic document. The processor (210) may execute the viewer application (260) in response to an input for opening and / or browsing the file (280). The processor (210) executing the viewer application (260) may execute a rendering engine (251) included in the electronic document library (250) so that at least a portion of the electronic document (e.g., a portion corresponding to the shape and / or size of the display (110) and / or browser portion) is displayed. The electronic document library (250) may be a set of instructions for executing functions for processing PDF-based files (280). The electronic document library (250) may be referred to as a PDF solution.

[0055] In one embodiment, the processor (210) executing the rendering engine (251) may store an image (e.g., raster graphics) representing at least a portion of the electronic document in the electronic document view (252) for storing the rendering result of the electronic document. By executing the rendering engine (251), the processor (210) may obtain information (e.g., objects) defined based on PDF from the file (280). Using the obtained information, the processor (210) may render the electronic document.

[0056] In one embodiment, the processor (210) may execute an electronic document manager (253) to control the creation, display, and / or storage of electronic documents (e.g., control the rendering engine (251) and / or the electronic document analyzer (254)). For example, the electronic document manager (253) may be included in the electronic document library (250) as an interface for providing functions of the electronic document library (250). By executing the electronic document manager (253), the processor (210) may control the execution of the rendering engine (251) and / or the electronic document analyzer (254). For example, the processor (210) may execute the electronic document analyzer (254) using data obtained from the rendering engine (251). By executing the electronic document manager (253), the processor (210) can convert or provide the analysis results of the electronic document analyzer (254) into information usable (or readable) by an external device (e.g., a viewer application (260)). For example, the processor (210) can execute the electronic document manager (253) to store (cach) at least a portion of the electronic document in the memory (220) (e.g., volatile memory). Based on the caching of the electronic document, the processor (210) can perform operations related to the electronic document more quickly.

[0057] The processor (210) that identifies the electronic document represented by the file (280) may support a function (e.g., a summary function and / or a translation function) for generating content related to the electronic document. For example, the processor (210) may execute a summary service (261) of the viewer application (260) to obtain a text summarizing the content of the electronic document. The operation of summarizing the electronic document may include the operation of generating at least one natural language sentence describing the content. The at least one natural language sentence generated by the operation of summarizing the electronic document may have a length and / or size less than the length and / or size of the content of the electronic document. For example, the processor (210) may execute a translation service (262) of a viewer application (260) to execute a translation function for at least a portion of the content of an electronic document (e.g., text located between visual objects (141, 142) within an electronic document (130) displayed on a display (110) of FIG. 1). The processor (210) may execute the translation service (262) using text of the electronic document selected by user input.

[0058] In order to support the summary function and translation function based on the summary service (261) and the translation service (262), respectively, the processor (210) may execute an artificial intelligence model (270). The artificial intelligence model (270) is a computational model designed to simulate human neural activities (e.g., reasoning activities, creative activities, cognitive activities), and may include a program, hardware (e.g., NPU and / or GPU), or any combination thereof for performing calculations represented by the computational model. The artificial intelligence model (270) may be executed independently by the processor (210) and / or the electronic device (101) (e.g., on-device and / or stand-alone). The embodiment is not limited thereto, and the electronic device (101) may communicate with an external electronic device (e.g., a server) configured to execute the artificial intelligence model (270) to execute functions (e.g., summary functions and / or translation functions) based on the artificial intelligence model (270). To communicate with external electronic devices, the electronic device (101) may further include communication circuitry (and / or a modem).

[0059] For example, the processor (210) may execute a translation function based on an artificial intelligence model (270) (or a translation model (272)) based on a user input. For example, the processor (210) may execute a summary function based on an artificial intelligence model (270) (or a summary model (271)) based on another user input. When executing the translation function, the processor (210) may display a translation result obtained from the translation model (272) on the display (110). The processor (210) may display the translation result on the display (110) based on the execution of the rendering engine (251).

[0060] In one embodiment, by executing the electronic document analyzer (254), the processor (210) may transform information related to an electronic document represented by a file (280) (e.g., information for rendering the electronic document based on the rendering engine (251)) into a form suitable for processing user input related to the electronic document (e.g., input for selecting at least a portion of text of the electronic document). For example, the processor (210) may determine or generate a recognition order of text of the electronic document from the information. As described above with reference to FIG. 1, since the order of objects included in the file (280) is different from the recognition order, the processor (210) may execute the electronic document analyzer (254) to determine or generate an order of text and / or characters that is different from the order in which the contents of the electronic document are rendered on the display (110). Using the above-determined order, the processor (210) can process an input for at least partially selecting text in an electronic document. By processing the input, the processor (210) can visually highlight at least a portion of the text relative to the remaining portion on the display (110).

[0061] As described above, according to one embodiment, when rendering an electronic document represented by a PDF-based file (280), the electronic device (101) may group or rearrange the order of text included in the electronic document based on units such as lines and / or paragraphs. By rearranging the text, the electronic device (101) may identify natural text to be used for performing a summary function and / or a translation function. Determining the order of text in the electronic document may be performed by executing an electronic document analyzer (254).

[0062] In one embodiment, the processor (210) may execute the electronic document analyzer (254) substantially concurrently with executing the file (280) and / or the viewer application (260). For example, the processor (210) may execute the electronic document analyzer (254) independently (or concurrently) with the rendering engine (251) that executes to render the electronic document. The file (280) and / or the electronic document represented by the file (280) may be processed in a distributed manner by the electronic document analyzer (254) and the rendering engine (251). For example, the processor (210) may execute the electronic document analyzer (254) in the background. The processor (210) may preferentially process a portion of the electronic document represented by the file (280) using the electronic document analyzer (254).

[0063] For example, a portion of an electronic document contained in a particular (virtual) page displayed on the display (110) and / or a specified number of pages (e.g., 2 to 3 pages) including said particular page may be first processed by the electronic document analyzer (254). For example, to respond more quickly to user input for a portion of an electronic document displayed on the display (110), the processor (210) may first process said portion using the electronic document analyzer (254). The embodiment is not limited thereto, and the processor (210) may execute the electronic document analyzer (254) based on receiving an input for at least partially selecting text.

[0064] Hereinafter, with reference to FIG. 3, exemplary operations performed by the electronic device (101) and / or the processor (210) of FIG. 2 are described.

[0065] FIG. 3 illustrates a flowchart of the operation of an electronic device displaying an electronic document (360). The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operations described with reference to FIG. 3. The order in which the operations of FIG. 3 are performed is not limited to the order illustrated in FIG. 3. For example, the electronic device may perform the operations of FIG. 3 in a different order than the order illustrated in FIG. 3. For example, the electronic device may perform at least two of the operations of FIG. 3 substantially simultaneously.

[0066] Referring to FIG. 3, in operation (310), according to one embodiment, an electronic device may receive an input for executing a file (350) (or a file (280) of FIG. 2) representing an electronic document (360) (or an electronic document (130) of FIG. 1). The input may include an input for opening and / or executing the file (350). The input may be received through a viewer application (e.g., the viewer application (260) of FIG. 2) executed to display the electronic document (360). The embodiment is not limited thereto, and the input of operation (310) may be received through a software application (e.g., a file browser application) for browsing files and / or directories stored in a memory of the electronic device (e.g., the memory (220) of FIG. 2). For example, the electronic device executing the software application may display an icon representing the file (350) on the display. The input of the action (310) may include a mouse click (or double click) on the icon and / or a tap gesture.

[0067] Referring to FIG. 3, an embodiment is illustrated in which an electronic device receives an input for executing a file (350) including a first object (352-1) including a first text (361), a second object (352-2) including a second text (362), and a third object (352-3) including a third text (363). The electronic device receiving the input can perform an operation (320) based on the file (350) specified by the input.

[0068] Referring to FIG. 3, within operation (320), according to one embodiment, an electronic device may render objects (e.g., first object (352-1) to third object (352-3)) stored in a file (350) to display at least a portion of an electronic document (360) on a display (e.g., display (110) of FIG. 1 and / or FIG. 2). The file (350) may have a format readable by any viewer application that supports PDF.

[0069] Referring to FIG. 3, the structure of a PDF-based file (350) is schematically illustrated. The file (350) may include a header (351) (or header area), a body (352) (or body area), a cross-reference table (353) (or cross-reference table area and / or reference area), and / or a trailer (354) (or trailer area). Within the file (350), each of the header (351), the body (352), the cross-reference table (353), and the trailer (354) may be surrounded by a designated code and / or a designated identifier.

[0070] In one embodiment, areas of a file (350) may be configured to store information defining an electronic document (360). For example, within a header (351) of a file (350), a PDF version (or PDF standard) used to generate the file (350) and / or the electronic document (360) represented by the file (350) may be stored. For example, within a body (352) of a file (350), information representing one or more objects that constitute the content of the electronic document (360) (e.g., object information and / or content objects) may be stored. A PDF may be defined to store the content of an electronic document (360) as a set of information referred to as "objects." Based on the objects, the content of the electronic document (360) may be defined, such as text, shapes such as paths, images, and / or shadows (e.g., translucent shadows overlapping the shapes). An object for defining text may be referred to as a text object, and an object for defining a shape may be referred to as a path object. A text object may include information used for selecting text (e.g., selecting text for copying, pasting, summarizing, and / or translating). For example, objects (352-1, 352-2, 352-3) located in the body (352) of a file (350) may represent texts (361, 362, 363) included in an electronic document (360), respectively. The objects may be referred to as document objects and / or document elements.

[0071] For example, information indicating the locations of multiple objects included in the body (352) may be stored within the cross-reference table (353) of the file (350). For example, information indicating the location (or size) of the cross-reference table (353) within the file (350) and / or the top-level object among the multiple objects included in the body (352) may be stored within the trailer (354) of the file (350).

[0072] According to one embodiment, the electronic device may sequentially identify (e.g., parse) objects (352-1, 352-2, 352-3) included in a body (352) of a file (350) based on an input of an operation (310), and display texts (361, 362, 363) corresponding to each of the objects (352-1, 352-2, 352-3). Referring to FIG. 3, the electronic device may render or display an electronic document (360) including a first text (361), a third text (363) following the first text (361), and a second text (362) following the third text (363). The electronic device may identify a location of the first text (361) within the electronic document (360) using location information of the first object (352-1). Similarly, the electronic device can identify the location of the second text (362) within the electronic document (360) using the location information of the second object (352-2). Similarly, the electronic device can identify the location of the third text (363) within the electronic document (360) (e.g., the location between the first text (361) and the second text (362)) using the location information of the third object (352-3). Based on the identified locations, the electronic device can render the electronic document (360). The order in which the electronic device renders the texts (e.g., the first text (361) to the third text (363)) within the electronic document (360) may be related to the locations of the texts within the electronic document (360). For example, the electronic device may display (or render) the first text (361) positioned at the top of the electronic document (360) among the first text (361) to the third text (363), then display (or render) the third text (363) positioned after the first text (361) within the electronic document (360), and then display (or render) the second text (362) positioned after the third text (363) within the electronic document (360).

[0073] Referring to FIG. 3, when sequentially identifying objects (352-1, 352-2, 352-3) and placing texts (361, 362, 363) in an electronic document (360) (or a (virtual) page within the electronic document (360)), the order in which the texts (361, 362, 363) are placed within the electronic document (360) may be different from the order in which a user viewing the electronic document (360) reads (or recognizes) the texts (361, 362, 363). The display of the operation (320) may be performed by executing the rendering engine (251) of FIG. 2.

[0074] Referring to FIG. 3, in operation (330), according to one embodiment, the electronic device may group text included in an electronic document based on lines and / or paragraphs to identify an order to be used for selecting and / or segmenting the text. The order of operation (330) may differ from the order of objects stored in a file (350). For example, the electronic device may adjust the order of the objects. For example, the order of the objects may be adjusted to correspond to the order of texts (361, 362, 363) arranged in the electronic document (360).

[0075] In one embodiment, the electronic device may execute the electronic document analyzer (254) of FIG. 2 to perform the operation (330). An operation (330) to be used for selecting, extracting, and / or segmenting text based on user input may include a preprocessing operation (e.g., an operation of an electronic device described with reference to FIGS. 4 and / or 5 ) for segmenting texts (361, 362, 363) corresponding to objects (352-1, 352-, 352-3) respectively down to the word level (or character level), an operation for grouping text segmented by the preprocessing operation into one or more lines (e.g., an operation for an electronic device described with reference to FIG. 6 ), an operation for grouping groups of text based on one or more lines into one or more paragraphs (e.g., an operation for an electronic device described with reference to FIGS. 7 and / or 8 ), and an operation for determining an order using groups of text based on one or more paragraphs (e.g., an operation for an electronic device described with reference to FIGS. 8 and / or 9 ). To perform the operation (330), the electronic device may identify, from the file (350) (or body (352)), location data for the text, the style of the text (e.g., font size and / or line spacing), and other content of the electronic document (360) having a shape that segments the text (e.g., an object defining a path, representing a shape such as a table).

[0076] Referring to FIG. 3, in operation (340), according to one embodiment, an electronic device may display a visual object indicating that texts are selected in the identified order on an electronic document displayed on a display, based on an input for text. For example, within the electronic document (360) of FIG. 3, the electronic device may receive an input for a first text (361) and a third text (363). For example, the electronic device may identify or detect a drag input directed from the first text (361) to the third text (363). The electronic device receiving the input may display a visual object indicating that the first text (361) and the third text (363) are selected, based on the arrangement (or order) of the first text (361), the second text (362), and the third text (363) within the electronic document (360), instead of the order of the objects.

[0077] As described above, the electronic device can determine the order of the operations (330) to at least partially select text of the electronic document (360) based on the input of the operations (340), or can make the order of the objects (352-1, 352-2, 352-3) used to display the electronic document (360) identical to the order (or recognition order) of the texts (361, 362, 363) corresponding to the objects (352-1, 352-2, 352-3) in the electronic document (360). By using the order of the operations (330), the electronic device can more accurately respond to user input performed to select at least a portion of the electronic document (360).

[0078] Below, with reference to FIG. 4 and / or FIG. 5, a preprocessing operation of an electronic device performed to identify the order of operations (330) is described.

[0079] FIG. 4 illustrates an exemplary operation of an electronic device for segmenting text represented by objects of a file representing an electronic document (e.g., file (280) of FIG. 2 and / or file (350) of FIG. 3). The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operation described with reference to FIG. 4. The operation of FIG. 4 may be related to the operation (330) of FIG. 3. The operation of FIG. 4 may be performed by an electronic device executing the electronic document analyzer (254) of FIG. 2.

[0080] Referring to FIG. 4, a table (410) included in an electronic document (e.g., the electronic document (130) of FIG. 1 and / or the electronic document (360) of FIG. 3) is exemplarily illustrated. The table (410) may include lines defined by one or more objects (e.g., objects referred to as paths and / or path objects) included in a file representing the electronic document. By identifying objects (e.g., text objects) defining each of the texts (411, 412, 413, 414, 415, 416, 417) from the file, the electronic device may display the texts (411, 412, 413, 414, 415, 416, 417) on the table (410). The objects may include information (e.g., coordinate values) indicating the locations of texts (411, 412, 413, 414, 415, 416, 417) within the electronic document. The objects may include information indicating the style (e.g., font size including width and / or height, font name, kerning, line spacing, color, bold, underline, subscript, superscript, and / or italic) of the texts (411, 412, 413, 414, 415, 416, 417).

[0081] An electronic device that identifies objects representing each of the texts (411, 412, 413, 414, 415, 416, 417) may segment the texts (411, 412, 413, 414, 415, 416, 417) at a character level and / or a word level. For example, the electronic device may identify or determine one or more groups of characters based on at least one of whether a space between characters exceeds a specified threshold and / or whether a shape (e.g., a line of a table represented by a path object) is located in an area between characters.

[0082] For example, the electronic device may determine or calculate a designated threshold to be used for grouping characters using a font size and / or font name defined by objects. For example, the designated threshold may be determined based on the width of a character indicated by the font size and the font name (or font type). For example, the designated threshold may be determined as a multiple of an average value of the widths of characters included in the texts (411, 412, 413, 414, 415, 416, 417) (e.g., an average value Х 2). The embodiment is not limited thereto, and the designated threshold may be determined heuristically (or statistically) as an interval between apparently spaced words.

[0083] Referring to FIG. 4, the electronic device can check the spacing between characters along the direction in which the characters are aligned (or arranged, or written). For example, the electronic device can compare the spacing between adjacent characters in the horizontal direction (or row direction) with the specified threshold. When identifying a spacing below the specified threshold, the electronic device can assign characters having the spacing to one group. When identifying a spacing exceeding the specified threshold, the electronic device can assign characters having the spacing to different groups.

[0084] Referring to FIG. 4, groups of characters (421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433) are illustrated, which are identified by comparing the spacing between characters. The spacing between the groups (421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433) may be greater than a specified threshold. The spacing between characters in each of the groups (421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433) may be smaller than a specified threshold.

[0085] Referring to FIG. 4, characters included in texts (411, 412, 413, 414, 415, 416, 417) can be segmented by a table (410) including a plurality of lines. Hereinafter, with reference to FIG. 5, an exemplary operation of an electronic device that groups characters according to the positional relationship between lines and characters included in an electronic document is described.

[0086] FIG. 5 illustrates exemplary operations of an electronic device for determining a group of texts based on spacing between the texts. The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operations described with reference to FIG. 5. The operations of FIG. 4 may be related to the operations (330) of FIG. 3. The electronic device may execute the electronic document analyzer (254) of FIG. 2 to perform the operations of FIG. 5.

[0087] Referring to FIG. 5, a table (500) including exemplary cases for grouping exemplary texts (e.g., first text to tenth text) represented by objects is illustrated. According to one embodiment, an electronic device may group characters represented by a text object representing a text and a path object representing a shape by utilizing a positional relationship between the text object and the path object representing a shape. An electronic device that identifies one or more path objects from a file representing an electronic document (e.g., file (280) of FIG. 2 and / or file (350) of FIG. 3) may merge and / or filter the one or more path objects to identify a path object to be used for checking the positional relationship.

[0088] For example, an electronic device that identifies multiple path objects from a file may determine whether to use each of the multiple path objects for grouping text based on the position, line width, width, and / or height of the multiple path objects. For example, if the entire width and height defined by the path object are less than or equal to a specified threshold, the electronic device may determine the path object as a dot (or noise). For example, if the distance between the path object and the text object is a multiple (e.g., 2 times) of the average of the widths of the text objects, the electronic device may determine the path object as a path object to be compared with the text object. A path object determined to be a dot may not be used for grouping text. The electronic device may merge path objects that at least partially overlap each other within the electronic document into a single path object. For example, the electronic device may group text using the merged path object. By determining whether each of the path objects is to be used for grouping text, the electronic device can determine or filter some of the path objects to be used for grouping text.

[0089] The electronic device can identify the positional relationship between the path object and the text object by using the area information of the path object and the text object (e.g., the x-coordinate values ​​of the starting point and the ending point, and the y-coordinate values). For example, when the path object is interposed between text objects, the electronic device can determine whether to merge characters (or words) included in the text objects based on the positional relationship between the path object and the text objects.

[0090] For example, as in the first case, if a line (511) represented by a path object is interposed between first text and second text, respectively represented by text objects, and the height of the line (511) exceeds a value obtained by applying a specified ratio (e.g., 50%) to the average height (or maximum height) of characters included in the text objects, the electronic device may not merge the first text and the second text into one group. For example, the first text and the second text may be assigned to different groups.

[0091] For example, as in the second case, if the distance (h2) between the horizontal axes of the third text and the fourth text exceeds the threshold distance, the electronic device may assign the third text and the fourth text to different groups independently of the line (521) intervening between the third text and the fourth text. Even if the height of the line (521) is less than or equal to a value obtained by applying a specified ratio to the average height (or maximum height) of characters included in the text objects, the electronic device may not include the third text and the fourth text in the same group.

[0092] For example, as in the third case, when a line (531) disposed between the fifth text and the sixth text along the horizontal axis (β) does not intersect the horizontal axis (β), the electronic device can group the fifth text and the sixth text independently of the line (531) interposed between the fifth text and the sixth text.

[0093] For example, as in the fourth case, a path object may represent an icon (541). The electronic device may determine that the path object represents an icon (e.g., icon (541)) if the ratio between the width and height of the path object is within a specified range of 0.8 to 1.2. The electronic device, which has identified the icon (541) represented by the path object, may divide the characters and / or texts distinguished by the icon (541) into different groups. Referring to the fourth case of FIG. 5, the electronic device may assign texts located on the left side of the phone-shaped icon (541) (“Inquiry” and “Matters”) to one group. The electronic device may assign texts located on the right side of the icon (541) (“(02) 123-4567”) to one group, independently of the texts. For example, a group of text (“Contact Us”) may be created to the left of the icon (541), and another group of text (“(02) 123-4567”) may be created to the right of the icon (541).

[0094] For example, in the fifth and / or sixth cases, when no shape is positioned between the texts, the electronic device may determine whether to merge the texts by using the spacing between the texts. For example, for the seventh text and the eighth text that are spaced apart by the spacing (d5), the electronic device may determine whether to merge the seventh text and the eighth text by comparing the spacing (d5) with a threshold spacing. For example, when the spacing (d5) is less than the threshold spacing, the electronic device may add the seventh text and the eighth text to one group. For example, in the sixth case, when the spacing (d6) between the ninth text and the tenth text is greater than or equal to the threshold spacing, the electronic device may assign the ninth text and the tenth text to different groups, respectively.

[0095] For example, as in the seventh case, the electronic device can group text using lines (573) within the table. The electronic device can group text only using lines (573) represented by path objects. For example, the character (572) representing the line may not be used to group text or separate text into different groups. Referring to the seventh case, text including the character (572) (“Department of Korean Language and Literature | 1”) may be included in one group. Referring to the seventh case, texts distinguished by lines (573) forming the table (571) (e.g., “1” located in the same row as “Business School” and “Business School”) may be assigned to different groups or separated by the lines (573).

[0096] Within the first to seventh cases, exemplary operations for grouping texts of a language arranged along the horizontal direction of an electronic document have been described, but the embodiments are not limited thereto. For example, if the text of the electronic document is arranged along the vertical direction (e.g., Japanese), the electronic device may check the spacing between text and / or characters along the vertical direction. Based on the checked spacing, the electronic device may group the text along the vertical direction. Within the above example, the electronic device may check or determine whether the text is segmented along a horizontal path. When comparing the characters of the language arranged vertically with a line of a path object (e.g., lines (511, 521, 531, 573)), the electronic device may compare the width of the line with a value obtained by applying a specified ratio (e.g., 50%) to the average width (or maximum width) of the characters to determine whether to assign adjacent characters to different groups.

[0097] In one embodiment, an electronic document may include both vertically arranged text and horizontally arranged text. For example, an electronic device that identifies an electronic document including a first paragraph including vertically arranged text and a second paragraph including horizontally arranged text may scan the text along the horizontal direction to determine whether the text can be grouped along the horizontal direction. In the above example, the electronic device may group the text of the second paragraph along the horizontal direction. In the second paragraph, the electronic device that identifies the text grouped along the horizontal direction may perform grouping for the remaining text within the second paragraph along the horizontal direction. In the above example, the electronic device may perform grouping for the text within the first paragraph that cannot be grouped along the horizontal direction along the vertical direction. If the electronic device cannot group the text within the remaining text within the first paragraph along the horizontal direction, the electronic device may perform grouping for the remaining text in the vertical direction.

[0098] An electronic device may be capable of identifying or verifying the orientations of text included in each paragraph of an electronic document. If the electronic document includes a first paragraph including text arranged vertically and text arranged horizontally, the electronic device may generate or store an attribute corresponding to the first paragraph, indicating that the text included in the first paragraph is arranged vertically. Similarly, the electronic device may generate or store an attribute for a second paragraph, indicating that the text included in the second paragraph is arranged horizontally. The attributes may be used to determine text to be sequentially selected based on an input for selecting text within the electronic document when the input is received.

[0099] As described above with reference to FIGS. 4 and 5, an electronic device that detects text objects each representing text can group texts corresponding to each of the text objects using the positions of the texts within an electronic document. The grouping can be performed independently of the mapping between text objects and characters. The electronic device can identify positional relationships within the electronic document between characters represented by the text objects and shapes represented by the path objects using positional information of the text objects and path objects. Using the positional relationships, the electronic device can group the characters (e.g., groups (421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433)).

[0100] Hereinafter, with reference to FIG. 6, an exemplary operation of an electronic device for identifying groups of texts (e.g., line groups) corresponding to a specific line (or specific row) by using groups of characters (or texts) obtained by the operations of FIGS. 4 and 5 is described. For example, the electronic device may group the groups down to the line level and / or paragraph level to identify or determine groups of texts suitable for translation and / or summarization.

[0101] FIG. 6 illustrates an exemplary operation of an electronic device for identifying groups of texts based on spacing between characters. The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operation described with reference to FIG. 6. The operation of FIG. 6 may be related to the operation (330) of FIG. 3. An electronic device executing the electronic document analyzer (254) of FIG. 2 may perform the operation of FIG. 6 on text of an electronic document.

[0102] Referring to FIG. 6, an exemplary result of performing the preprocessing operation described with reference to FIGS. 4 and 5 on texts of an electronic document (610) is illustrated. The dashed boxes displayed as overlapping lines on the electronic document (610) may each correspond to groups of characters grouped by the preprocessing operation. The electronic device may group the groups along the row direction of the electronic document (610) to identify or determine a group of characters (e.g., a line group) located on a single line.

[0103] To group (e.g., join, combine, and / or union) groups identified along a line direction, the electronic device may compare font sizes (e.g., widths and / or heights of characters) of characters included in each group. For example, the electronic device may use the similarity of the font sizes of the characters included in each group to determine whether to group the groups into one line group. For example, the electronic device may group the groups into one line group if the difference in the font sizes is less than or equal to a value obtained by applying a specified percentage (e.g., 7%) to the maximum font size of two font sizes.

[0104] For example, when comparing heights of English letters, the electronic device may set a designated value (e.g., approximately 0.67) based on the height difference between the letters of the alphabet (e.g., the height difference between the letters Y and a) as an error range. If the height differences of the letters included in groups adjacent to each other along the row direction fall within the error range, the electronic device may combine the groups to identify or create a single line group. A representative value of the heights of the letters included in the line group may be set to an average value and / or a maximum value of the heights. The electronic device may calculate differences between the letters of another group adjacent to the line group along the row direction and the representative value. If the calculated differences fall within the error range, the electronic device may add the other group to the line group. When the other group is added to the line group, the electronic device may reset the representative value of the heights of the letters included in the line group. Based on the reset, the electronic device may check another group adjacent to the line group along the row direction. Based on repeated comparisons, the electronic device can scan all groups located in a particular row.

[0105] When merging groups along a line direction, the electronic device can compare the spacing between groups (e.g., horizontal spacing) with a threshold to determine whether to group the groups into a single line group. The threshold can be set as in mathematical expression 1.

[0106]

[0107] The value e in mathematical expression 1 is an empirically determined value, and the embodiment is not limited thereto. The similarity of mathematical expression 1 can be set as in mathematical expression 2. The reference value of mathematical expression 1 can be set as in mathematical expression 3.

[0108]

[0109]

[0110] The first group and the second group of mathematical expressions 2 and / or 3 may correspond to groups of characters that are adjacent to each other along the row direction, respectively. The representative height of mathematical expressions 2 and / or 3 may be the maximum height and / or the average height of the heights of characters included in the corresponding group. The constant 0.67 of mathematical expression 2 may be empirically determined. The embodiment is not limited thereto. The similarity, reference value, and / or threshold of mathematical expressions 1 to 3 may be set differently depending on the language.

[0111] For example, if the gap between groups is less than or equal to the threshold of Equation 1, the electronic device may merge or group the groups into a single line group. If the gap between groups exceeds the threshold of Equation 1, the electronic device may assign the groups to different line groups, or may not merge them.

[0112] Referring to FIG. 6, exemplary line groups determined by grouping characters of an electronic document (610) are illustrated. Solid boxes displayed as overlapping lines on the electronic document (610) may correspond to the line groups, respectively. By grouping groups of characters along a row direction, an electronic device may identify one or more line groups that are presumed to be recognized by a user as characters arranged on a single line. Having identified multiple line groups, the electronic device may group the multiple line groups to determine at least one paragraph group, each corresponding to a different paragraph of the electronic document.

[0113] Hereinafter, with reference to FIG. 7, an exemplary operation of an electronic device for identifying one or more paragraph groups based on the positional relationship between line groups included in an electronic document is described.

[0114] FIG. 7 illustrates an exemplary operation of an electronic device based on spacing (or distances) between groups of text. The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operation described with reference to FIG. 7. The operation of FIG. 7 may be related to the operation (330) of FIG. 3. The electronic device may execute the electronic document analyzer (254) of FIG. 2 to perform the operation of FIG. 7.

[0115] Referring to FIG. 7, a plurality of line groups (ℓ1, ℓ2, ℓ3, ℓ4, ℓ5, ℓ6, ℓ7, ℓ8) identified from an exemplary electronic document (130) are exemplarily illustrated. The electronic device can identify or determine the plurality of line groups (ℓ1, ℓ2, ℓ3, ℓ4, ℓ5, ℓ6, ℓ7, ℓ8) for the electronic document (130) by performing the operations of FIGS. 4 to 6. Only eight line groups (ℓ1, ℓ2, ℓ3, ℓ4, ℓ5, ℓ6, ℓ7, ℓ8) are exemplarily illustrated for convenience of explanation, and the electronic device can also identify or generate one or more line groups for the texts of the remaining portion of the electronic document (130) by performing the operations of FIGS. 4 to 6.

[0116] Referring to FIG. 7, the electronic device can identify a dividing point (or reference point) capable of dividing the electronic document (130) along designated axes (e.g., the x-axis corresponding to the row direction and / or the y-axis corresponding to the column direction) of the electronic document (130) within the blank area between the line groups. Since the dividing point is determined in the blank area between the line groups, no dividing point is generated between the at least partially overlapping line groups (ℓ3, ℓ4, ℓ5). The dividing point can be formed when the width and / or height of the blank area formed along the designated axes exceeds a designated size.

[0117] Referring to FIG. 7, x-axis dividing points (△) determined based on the spacing of line groups in the x-axis direction and y-axis dividing points (O) determined based on the spacing of line groups in the y-axis direction are illustrated. The x-axis dividing points (△) indicate that an electronic document (130) can be at least partially divided in the y-axis direction based on the dividing points. The y-axis dividing points (O) indicate that an electronic document (130) can be at least partially divided in the x-axis direction based on the dividing points. An interval and an axis used for determining the dividing points can be mapped to each of the dividing points.

[0118] Having identified the exemplary dividing points of FIG. 7, the electronic device can cluster (or group) line groups using the intervals between the dividing points. For example, the electronic device can determine a boundary for dividing paragraphs based on the dividing points. The electronic device can separate a plurality of paragraphs using dividing points corresponding to one axis among the x-axis dividing points (△) and the y-axis dividing points (O), and then further separate the separated paragraphs using dividing points corresponding to the other axis. Hereinafter, an exemplary operation of the electronic device for identifying a plurality of paragraphs using dividing points is described.

[0119] For example, an electronic device may cluster (or group) segmentation points using intervals set for each of a plurality of segmentation points. For example, if the similarity of the intervals is less than a specified similarity, the electronic device may group the segmentation points corresponding to the intervals into one cluster (or group). For example, if a representative value (e.g., a maximum interval and / or an average interval) of the intervals of the segmentation points included in a specific cluster is B, and the interval of the segmentation point to be compared with the specific cluster is x, the electronic device may calculate the reference value of Equation 4.

[0120]

[0121] If the reference value of Equation 4 is within the range of -0.0725 to 0.0725, the electronic device may add the segmentation point to the specific cluster. The constants (0.0725) are empirically determined and may vary depending on the embodiment.

[0122] After clustering the segmentation points, the electronic device can sort the representative values ​​of the intervals of the clusters in descending order. For the sorted clusters, the electronic device can determine or segment paragraphs of the electronic document (130) based on the segmentation points of the clusters having representative values ​​exceeding a threshold interval.

[0123] Depending on the number of dividing points included in the cluster, the electronic device can determine whether to divide the paragraph. For example, if the number of x-axis dividing points (△) included in the cluster is 1 or more, the electronic device can divide the paragraph of the electronic document (130) based on the x-axis dividing points (△). For example, if the number of y-axis dividing points (O) included in the cluster is 3 or more, the electronic device can divide the paragraph of the electronic document (130) based on the y-axis dividing points (O). If dividing points corresponding to different axes are included in one cluster, the electronic device can divide the paragraphs based on the operation of FIG. 8.

[0124] Since the splitting points are divided into x-axis splitting points (△) and y-axis splitting points (O), one cluster may include splitting points corresponding to different axes. If a cluster includes splitting points of a single axis, the electronic device may split a paragraph based on the splitting points. For example, based on the y-axis splitting point (751) having the largest gap, the electronic device may split the electronic document (130) along a direction parallel to the x-axis, thereby identifying a paragraph (711).

[0125] As described above, an electronic device that identifies a paragraph group using splitting points may compare representative values ​​of font sizes of each of the line groups within the paragraph group (e.g., maximum height and / or average height of characters included in the line group) to determine whether to further split the paragraph group. For example, the electronic device may determine, for a paragraph including a plurality of line groups, whether to maintain the plurality of line groups as a single paragraph group using the similarity, threshold, and reference values ​​of Equations 1 to 3 for the plurality of line groups. For example, the first group and the second group of Equations 1 to 3 may correspond to each of the line groups included in the paragraph group.

[0126] Referring to FIG. 7, the results of identifying paragraph groups (711, 712, 713, 714, 715, 716) of an electronic document (130) by performing the operations of FIGS. 7 to 8 are illustrated. Since not only the division points but also the font size (e.g., height) of the line group is utilized, the line group (ℓ2) can be determined as a different paragraph group (713) from the paragraph group (712) of the line groups (ℓ3, ℓ4, ℓ5). The font size can be set differently depending on the position of the text within the electronic document (130), such as a title, a subtitle, a header, a footer, an image caption, and / or the body. Considering the above characteristics, the electronic device can divide the paragraph groups (711, 712, 713, 714, 715, 716) by further utilizing the font size as well as the spacing between the line groups.

[0127] An electronic device that has identified paragraph groups (711, 712, 713, 714, 715, 716) can determine the order of the paragraph groups (711, 712, 713, 714, 715, 716) according to a conventional order of reading (or recognizing) text of an electronic document (130). In addition, the electronic device can determine the order of line groups included in each of the paragraph groups (711, 712, 713, 714, 715, 716) and characters included in each of the line groups. The finally determined order can be used to determine at least a portion of the text to be selected based on an input for at least partially selecting the text of the electronic document (130), as described above with reference to FIG. 1.

[0128] FIG. 8 illustrates an exemplary operation of an electronic device that groups line groups to identify one or more paragraph groups. The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operation described with reference to FIG. 8. The operation of FIG. 8 may be related to the operation (330) of FIG. 3. The electronic device may execute the electronic document analyzer (254) of FIG. 2 to perform the operation of FIG. 8.

[0129] Referring to FIG. 8, a table (800) including exemplary cases for determining x-axis dividing points (△) and y-axis dividing points (O) is illustrated. Referring to a first case of the table (800), two x-axis dividing points (△) having the same spacing (x1) and one y-axis dividing point (O) are illustrated. For dividing points having the same length, the electronic device can group paragraphs based on a relatively small number of types of dividing points. For example, since there is one y-axis dividing point (O) and two x-axis dividing points (△), the electronic device can identify or determine a paragraph group (811) including a first text, a second text, and a third text, and a paragraph group (812) including a fourth text, a fifth text, and a sixth text, by dividing the paragraphs based on the y-axis dividing points.

[0130] When the intervals of the dividing points are different, the electronic device can create or determine a paragraph group based on the dividing point having a relatively large interval. Referring to the second case of the table (800), since the interval (x2) of the x-axis dividing point (△) is the largest among the intervals (x2, x3, y1) of the three dividing points, the electronic device can group two paragraph groups (821, 822) based on the x-axis dividing point (△) having the interval (x2). The paragraph group (821) can include the seventh text and the tenth text, which are located on the left side of the x-axis dividing point (△) having the interval (x2). The paragraph group (822) can include the eighth text, the ninth text, the eleventh text, and the twelfth text, which are located on the right side of the x-axis dividing point (△) having the interval (x2).

[0131] Similarly, referring to the third case of the table (800), among the intervals (x4, y2) of the three dividing points, since the interval (y2) of the y-axis dividing point (O) is relatively large, the electronic device can determine the paragraph group (831) including the 13th text, the 14th text, and the 15th text, and the paragraph group (832) including the 16th text, the 17th text, and the 18th text based on the y-axis dividing point (O).

[0132] FIG. 9 illustrates the operation of an electronic device for determining the order of groups of text included in an electronic document. The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operation described with reference to FIG. 9. The operation of FIG. 9 may be related to the operation (330) of FIG. 3. The electronic device may execute the electronic document analyzer (254) of FIG. 2 to perform the operation of FIG. 9.

[0133] Referring to FIG. 9, cases in which groups of electronic documents (e.g., paragraph groups (911, 912, 913, 914, 921, 922, 923, 924)) are determined based on the operations described above with reference to FIGS. 4 to 8 are included in the table (900). Depending on the character, the direction in which the character is read (e.g., the direction in which the character is recognized) may be defined differently. For example, English, Korean, Japanese (if not written vertically), and Chinese may be defined to write characters sequentially from left to right along the horizontal direction. For example, Arabic and Hebrew may be defined to write characters sequentially from right to left along the horizontal direction. An electronic device may identify characters included in an electronic document based on a range of binary codes. For example, for hexadecimal binary code, the first range of 0x00690 or greater and 0x008FF or less, the second range of 0x0FB1D or greater and 0x0FDFF or less, the third range of 0x0FE70 or greater and 0x0FEFF or less, the fourth range of 0x10800 or greater and 0x10CFF or less, and the fifth range of 0x1E800 or greater and 0x1EEFF or less can be mapped to characters representing Arabic and / or Hebrew.

[0134] According to one embodiment, the electronic device may determine, when identifying the binary code of the first to fifth ranges described above from an electronic document (e.g., when identifying from a text object of a file representing the electronic document), that the electronic document includes characters of a language defined to be read from right to left. Based on the determination, the electronic device may determine the order of paragraph groups according to the right to left direction and / or the top to bottom direction. Referring to the second case of table (900), the paragraph group (921) located at the top of the electronic document may have an order that is higher than the other paragraph groups (922, 923, 924), and the paragraph group (922) may have an order that is higher than the remaining paragraph groups (923, 924) excluding the paragraph group (921). Among the paragraph groups (923, 924), the paragraph group (924) located on the right may have an order that is higher than the paragraph group (923).

[0135] In one embodiment, if the electronic device fails to identify any binary codes included in the first to fifth ranges from the electronic document, the electronic device may determine that the electronic document includes characters of a language defined to be read from left to right. Based on the determination, the electronic device may determine the order of paragraph groups according to the left to right direction and / or the top to bottom direction. Referring to the first case of table (900), the paragraph group (911) located at the top of the electronic document may have an order that is higher than the other paragraph groups (912, 913, 914), and the paragraph group (912) may have an order that is higher than the remaining paragraph groups (913, 914) excluding the paragraph group (911). Among the paragraph groups (913, 914), the paragraph group (913) located at the left may have an order that is higher than the paragraph group (914). Within each of the paragraph groups (911, 912, 913, 914), among the first character and the second character located to the right of the first character, the first character may have a precedence order over the second character.

[0136] As described above, according to one embodiment, an electronic device can group characters included in an electronic document at the line level or paragraph level, and then determine the order of the characters by further considering the reading direction of the characters. Using the determined order, the electronic device can accurately respond to a user input selecting at least a portion of the text included in the electronic document. For example, the user input indicates only the start and end points of the text range that the user wishes to select. The electronic device can identify a plurality of characters between the start and end points using the determined order. Using the identified plurality of characters, the electronic device can support functions ranging from copying and / or searching to functions utilizing an artificial intelligence model, such as summarizing and / or translating. For example, using the identified plurality of characters, the electronic device can generate a prompt for controlling the summarizing model. For example, by inputting the identified plurality of characters into a translation model (e.g., the translation model (272) of FIG. 2), the electronic device can execute a translation function.

[0137] FIG. 10 illustrates an exemplary operation of an electronic device (101) displaying an electronic document (1010). Referring to FIG. 10, an exemplary state of an electronic device (101) displaying an electronic document (1010) is illustrated. The electronic device (101) may display at least a portion of the electronic document (1010) on at least a portion (e.g., portion (122)) of the display (110) based on the operation described with reference to FIG. 1.

[0138] Within a file representing an electronic document (1010), information for displaying paragraphs included in the electronic document (1010) may be stored in an order different from the order in which the paragraphs are perceived by a human. For example, within the file, an object corresponding to a first paragraph (1011) may be stored before an object corresponding to a second paragraph (1012), and an object corresponding to a second paragraph (1012) may be stored before an object corresponding to a third paragraph (1013). A human will first view (or read) the second paragraph (1012) among the first paragraph (1011), the second paragraph (1012), and the third paragraph (1013), and then view the first paragraph (1011) after reading the second paragraph (1012). A human seeing the first paragraph (1011) will (conventionally) see the third paragraph (1013) among the second paragraph (1012) or the third paragraph (1013).

[0139] According to one embodiment, the electronic device (101) can estimate or obtain an order in which objects representing the first paragraph (1011) to the third paragraph (1013) are to be recognized by a human, by segmenting and / or grouping objects included in the electronic document (1010), as described above with reference to FIGS. 1 to 9. Using the obtained order, the electronic device (101) can process an input for selecting at least a portion of the electronic document (1010).

[0140] Referring to FIG. 10, an exemplary state of an electronic device (101) that receives a drag gesture from a first point (1021) to a second point (1023) on a display (110) displaying an electronic document (1010) is illustrated. The drag gesture may be performed by a user to select text between a word (e.g., “but”) displayed at the first point (1021) and a word (e.g., “made”) displayed at the second point (1023) within the electronic document (1010). The electronic device (101) that receives the drag gesture may select at least a portion of the text included in the electronic document (1010) by utilizing a result of estimating the order in which objects in the electronic document (1010) are recognized by a human.

[0141] For example, the electronic device (101) may display a visual object representing a selection of a portion of text in a second paragraph (1012) beginning with the word indicated at the first point (1021) (e.g., "However, advances in computer networks have made it much more convenient to distribute electronic documents than printed ones. Electronic display technology has made it possible to print them"), a portion of text in a first paragraph (1011) (e.g., "However, using electronic documents as final versions instead of paper has created problems with incompatible file formats"), and a portion of text in a third paragraph (1013) (e.g., "However, using electronic documents as final versions instead of paper has created problems with incompatible file formats"). The visual object may include a visual effect that visually emphasizes the text with respect to the remaining text in the electronic document (1010).

[0142] Meanwhile, when processing a user input including the drag gesture based on the order of objects included in a file representing an electronic document (1010), since the first point (1021) of the drag gesture is positioned on a word (e.g., “but”) included in the second paragraph (1012) and the second point (1023) of the drag gesture is positioned on a word (e.g., “made”) included in the third paragraph (1013), the electronic device (101) can highlight only a portion of the text of the second paragraph (1012) excluding the first paragraph (1011) (e.g., “However, the development of computer networks has made it much more convenient to distribute electronic documents than printed documents. Electronic display technology prints them”) and a portion of the text of the third paragraph (1013) (e.g., “However, using electronic documents as final versions instead of paper has created problems with incompatible file formats”).

[0143] Referring to FIG. 10, in an exemplary state of an electronic device (101) that receives a drag gesture from a first point (1021) to a second point (1023) on a display (110) displaying an electronic document (1010), the electronic device (101) may display a menu (150) indicating executable functions using text selected by the drag gesture. When a user selects one of the items in the menu (150), the electronic device (101) may execute a function corresponding to the selected item.

[0144] For example, an electronic device (101) that detects an input for selecting an item having the text "summary" can execute a summary function based on the text selected by the drag gesture (e.g., "However, the development of computer networks has made it much more convenient to distribute electronic documents than printed documents. Electronic display technology has made it possible to display them on a screen rather than print them. However, using electronic documents as final versions instead of paper has created the problem of incompatible file formats."). The electronic device (101) that executes the summary function can obtain or output another text having the content of the text but having a length less than or equal to the length of the text.

[0145] Meanwhile, if the electronic device (101) processes the drag gesture using only the order of objects in the file representing the electronic document (1010), the first paragraph (1011) will not be selected, and only a portion of the text of the second paragraph (1012) (e.g., "However, the development of computer networks has made it much more convenient to distribute electronic documents than printed documents. Electronic display technology prints them") and a portion of the text of the third paragraph (1013) (e.g., "However, using electronic documents as final versions instead of paper has created problems with incompatible file formats.") will be selected, so that the summary function and / or the translation function will be performed incompletely.

[0146] For another example, an electronic device (101) that detects an input for selecting an item having the text "translate" can execute a translation function based on the text selected by the drag gesture (e.g., "However, the development of computer networks has made it much more convenient to distribute electronic documents than printed documents. Electronic display technology has made it possible to display them on a screen rather than print them. However, using electronic documents as final versions instead of paper has created the problem of incompatible file formats."). The electronic device (101) that executes the translation function can obtain another text in a second language (e.g., English) from the text in a first language (e.g., Korean).

[0147] FIG. 11 is a block diagram of an electronic device (1101) within a network environment (1100) according to various embodiments. Referring to FIG. 11, in the network environment (1100), the electronic device (1101) may communicate with the electronic device (1102) via a first network (1198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (1104) or the server (1108) via a second network (1199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (1101) may communicate with the electronic device (1104) via the server (1108). According to one embodiment, the electronic device (1101) may include a processor (1120), a memory (1130), an input module (1150), an audio output module (1155), a display module (1160), an audio module (1170), a sensor module (1176), an interface (1177), a connection terminal (1178), a haptic module (1179), a camera module (1180), a power management module (1188), a battery (1189), a communication module (1190), a subscriber identification module (1196), or an antenna module (1197). In some embodiments, the electronic device (1101) may omit at least one of these components (e.g., the connection terminal (1178)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1176), camera module (1180), or antenna module (1197)) may be integrated into a single component (e.g., display module (1160)).

[0148] The processor (1120) may, for example, execute software (e.g., a program (1140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1101) connected to the processor (1120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1120) may store commands or data received from other components (e.g., a sensor module (1176) or a communication module (1190)) in a volatile memory (1132), process the commands or data stored in the volatile memory (1132), and store result data in a non-volatile memory (1134). According to one embodiment, the processor (1120) may include a main processor (1121) (e.g., a central processing unit or an application processor) or an auxiliary processor (1123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1121). For example, when the electronic device (1101) includes the main processor (1121) and the auxiliary processor (1123), the auxiliary processor (1123) may be configured to use less power than the main processor (1121) or to be specialized for a given function. The auxiliary processor (1123) may be implemented separately from the main processor (1121) or as a part thereof.

[0149] The auxiliary processor (1123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (1160), a sensor module (1176), or a communication module (1190)) of the electronic device (1101), for example, on behalf of the main processor (1121) while the main processor (1121) is in an inactive (e.g., sleep) state, or together with the main processor (1121) while the main processor (1121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1180) or a communication module (1190)). In one embodiment, the auxiliary processor (1123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0150] The memory (1130) can store various data used by at least one component (e.g., the processor (1120) or the sensor module (1176)) of the electronic device (1101). The data can include, for example, software (e.g., the program (1140)) and input data or output data for commands related thereto. The memory (1130) can include a volatile memory (1132) or a non-volatile memory (1134).

[0151] The program (1140) may be stored as software in memory (1130) and may include, for example, an operating system (1142), middleware (1144), or an application (1146).

[0152] The input module (1150) can receive commands or data to be used in a component of the electronic device (1101) (e.g., a processor (1120)) from an external source (e.g., a user) of the electronic device (1101). The input module (1150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0153] The audio output module (1155) can output audio signals to the outside of the electronic device (1101). The audio output module (1155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0154] The display module (1160) can visually provide information to an external party (e.g., a user) of the electronic device (1101). The display module (1160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (1160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0155] The audio module (1170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (1170) can acquire sound through the input module (1150), output sound through the sound output module (1155), or an external electronic device (e.g., electronic device (1102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1101).

[0156] The sensor module (1176) can detect the operating status (e.g., power or temperature) of the electronic device (1101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0157] The interface (1177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1101) with an external electronic device (e.g., the electronic device (1102)). In one embodiment, the interface (1177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0158] The connection terminal (1178) may include a connector through which the electronic device (1101) may be physically connected to an external electronic device (e.g., the electronic device (1102)). In one embodiment, the connection terminal (1178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0159] The haptic module (1179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1179) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0160] The camera module (1180) can capture still images and videos. In one embodiment, the camera module (1180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0161] The power management module (1188) can manage the power supplied to the electronic device (1101). According to one embodiment, the power management module (1188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0162] A battery (1189) may power at least one component of the electronic device (1101). In one embodiment, the battery (1189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0163] The communication module (1190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1101) and an external electronic device (e.g., electronic device (1102), electronic device (1104), or server (1108)), and the performance of communication through the established communication channel. The communication module (1190) may operate independently from the processor (1120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1190) may include a wireless communication module (1192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1194) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1104) via a first network (1198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1192) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1196) to verify or authenticate the electronic device (1101) within a communication network such as the first network (1198) or the second network (1199).

[0164] The wireless communication module (1192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1192) may support various requirements specified in the electronic device (1101), an external electronic device (e.g., the electronic device (1104)), or a network system (e.g., the second network (1199)). According to one embodiment, the wireless communication module (1192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 164 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC implementation.

[0165] The antenna module (1197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1198) or the second network (1199), may be selected from the plurality of antennas by, for example, the communication module (1190). A signal or power may be transmitted or received between the communication module (1190) and an external electronic device via the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1197).

[0166] According to various embodiments, the antenna module (1197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0167] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0168] According to one embodiment, commands or data may be transmitted or received between the electronic device (1101) and an external electronic device (1104) via a server (1108) connected to a second network (1199). Each of the external electronic devices (1102, or 704) may be the same or a different type of device as the electronic device (1101). According to one embodiment, all or part of the operations executed in the electronic device (1101) may be executed in one or more of the external electronic devices (1102, 1104, or 708). For example, when the electronic device (1101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1101). The electronic device (1101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (1104) may include an Internet of Things (IoT) device. The server (1108) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (1104) or server (1108) may be included within the second network (1199). The electronic device (1101) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0169] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0170] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0171] The term "module" used in various embodiments of this document may include a unit implemented in hardware, and may be used interchangeably with terms such as logic, block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0172] Various embodiments of the present document may be implemented as software (e.g., a program (1140)) including one or more instructions stored in a storage medium (e.g., an internal memory (1136) or an external memory (1138)) readable by a machine (e.g., an electronic device (1101)). For example, a processor (e.g., a processor (1120)) of the machine (e.g., an electronic device (1101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0173] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0174] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added. The electronic device (1101) of FIG. 11 may be an example of the electronic device (101) of FIGS. 1 and 2.

[0175] In one embodiment, a method may be required to identify an input for selecting at least a portion of text contained in an electronic document in order to copy, translate, and / or summarize the text. In one embodiment, a method may be required to specify or select at least a portion of the text corresponding to the input, based on an order of the text displayed through a display, independently of the order of objects representing different portions of the text within a file representing the electronic document. According to an embodiment, as described above, an electronic device (e.g., the electronic device 101 of FIG. 1 and / or the electronic device 1101 of FIG. 11) may include a display (e.g., the display 110 of FIG. 1 and / or FIG. 2), a memory (e.g., the memory 220 of FIG. 2) including one or more storage media and storing instructions, and at least one processor (e.g., the processor 210 of FIG. 2) including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a first input to execute a file (e.g., file (280) of FIG. 2 and / or file (350) of FIG. 3) comprising a first object (e.g., first object (352-1) of FIG. 3) comprising a first text (e.g., first text (361) of FIG. 3), a second object (e.g., second object (352-2) of FIG. 3) comprising a second text (e.g., second text (362) of FIG. 3), and a third object (e.g., third object (352-3) of FIG. 3) comprising a third text (e.g., third text (363) of FIG. 3) positioned between the first text and the second text.The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the first text, the second text, and the third text based on the first input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a visual object within the electronic document indicating that the first text, the third text, and the second text are selected by adjusting the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text based on a second input received with respect to the first text and the second text. In one embodiment, the electronic device may identify an input for at least partially selecting the text included in the electronic document in order to copy, translate, and / or summarize the text. In one embodiment, the electronic device can specify or select at least a portion of the text corresponding to the input based on the order of the text displayed through the display, independently of the order of objects representing different portions of the text within a file representing the electronic document.

[0176] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a plurality of objects including the first object, the second object, and the third object. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to group characters represented by the plurality of objects along a row direction of the electronic document, and to identify a plurality of line groups corresponding to each of the plurality of objects. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to group the plurality of line groups using spacing between lines of the electronic document represented by each of the plurality of line groups, and to identify a plurality of paragraph groups corresponding to each of the plurality of line groups. When executed individually or collectively by the at least one processor, the electronic device may cause the electronic device to identify an order of the plurality of paragraph groups to be used in selection of text based on the second input, based on an arrangement of the plurality of paragraph groups within the electronic document.

[0177] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine the plurality of line groups by grouping the characters using a threshold based on character width.

[0178] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine a plurality of reference points within the blank areas between the plurality of line groups using whether the blank areas between the plurality of line groups are divisible along the row direction or a column direction perpendicular to the row direction. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to group the plurality of line groups using intervals between line groups within which each of the plurality of reference points interposes, and directions of the blank areas into which each of the plurality of reference points is divided.

[0179] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine an order of the plurality of paragraph groups such that, while displaying the first text, the second text, and the third text in a first language, the first paragraph group, which is defined to be read in a direction from left to right, a first paragraph group, which is positioned to the left of the electronic document, has a higher order than a second paragraph group, which is positioned to the right of the first paragraph group.

[0180] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine an order of the plurality of paragraph groups such that the second paragraph group has a higher order than the first paragraph group while displaying the first text, the second text, and the third text in a second language, the second text being defined to be read in a direction from right to left.

[0181] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display executable objects associated with the first text, the third text, and the second text while displaying the visual object. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a third input for selecting any one of the executable objects. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to execute a function associated with an executable object corresponding to the third input, using a combination of the first text and the second text, based on the third input.

[0182] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to detect the second input based on a point of contact on the display that moved from the first text to the second text.

[0183] In one embodiment, as described above, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions, when executed by an electronic device including a display, may cause the electronic device to receive a first input for executing a file including a first object including first text, a second object including second text, and a third object including third text positioned between the first text and the second text. The instructions, when executed by the electronic device, may cause the electronic device to display an electronic document including the first text, the second text, and the third text, based on the first input. The instructions, when executed by the electronic device, may cause the electronic device to display, within the electronic document, a visual object indicating that the first text, the third text, and the second text are selected by adjusting the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text, based on a second input received with respect to the first text and the second text.

[0184] For example, the instructions, when executed by the electronic device, may cause the electronic device to identify a plurality of objects including the first object, the second object, and the third object. The instructions, when executed by the electronic device, may cause the electronic device to group characters represented by the plurality of objects along a row direction of the electronic document, and to identify a plurality of line groups corresponding to each of the plurality of objects. The instructions, when executed by the electronic device, may cause the electronic device to group the plurality of line groups using spacing between lines of the electronic document represented by each of the plurality of line groups, and to identify a plurality of paragraph groups corresponding to each of the plurality of line groups. The instructions, when executed by the electronic device, may cause the electronic device to identify an order of the plurality of paragraph groups to be used in selection of text based on the second input, based on an arrangement of the plurality of paragraph groups within the electronic document.

[0185] For example, the instructions, when executed by the electronic device, may cause the electronic device to determine the plurality of line groups by grouping the characters using a threshold based on the width of the characters.

[0186] For example, the instructions, when executed by the electronic device, may cause the electronic device to determine a plurality of reference points within the blank areas between the plurality of line groups using whether the blank areas between the plurality of line groups are divisible along the row direction or a column direction perpendicular to the row direction. The instructions, when executed by the electronic device, may cause the electronic device to group the plurality of line groups using intervals between line groups within which each of the plurality of reference points interposes, and directions of the blank areas divided at each of the plurality of reference points.

[0187] For example, the instructions, when executed by the electronic device, may cause the electronic device to determine an order of the plurality of paragraph groups such that, while displaying the first text, the second text, and the third text in a first language, the first text, the second text, and the third text are defined to be read in a direction from left to right, a first paragraph group, located on the left side of the electronic document, among the plurality of paragraph groups, has a higher order than a second paragraph group, located on the right side of the first paragraph group.

[0188] For example, the instructions, when executed by the electronic device, may cause the electronic device to determine an order of the plurality of paragraph groups such that the second paragraph group has a higher order than the first paragraph group while displaying the first text, the second text, and the third text in a second language, the second text being defined to be read in a direction from right to left.

[0189] For example, the instructions, when executed by the electronic device, may cause the electronic device to display executable objects associated with the first text, the third text, and the second text while displaying the visual object. The instructions, when executed by the electronic device, may cause the electronic device to receive a third input for selecting one of the executable objects. The instructions, when executed by the electronic device, may cause the electronic device to execute a function associated with the executable object corresponding to the third input, using a combination of the first text, the third text, and the second text, based on the third input.

[0190] For example, the instructions, when executed by the electronic device, may cause the electronic device to detect the second input based on a point of contact on the display that moved from the first text to the second text.

[0191] As described above, in one embodiment, a method of an electronic device including a display may be provided. The method may include receiving a first input for executing a file representing an electronic document. The method may include identifying objects within the file, which are groups of characters included in the electronic document, based on the first input, and displaying the electronic document on the display. The method may include determining a plurality of groups by grouping at least two characters positioned adjacent to each other within the electronic document using positional information of the characters represented by the objects. The method may include determining an order of the plurality of groups based on an arrangement of the plurality of groups within the electronic document. The method may include determining the one or more characters using the determined order of the plurality of groups based on receiving a second input for selecting one or more characters positioned between a first character and a second character among the characters.

[0192] For example, the method may include an action of superimposing a visual object corresponding to the second input on the first character, the one or more characters, and the second character on the display.

[0193] For example, the operation of determining the plurality of groups may include an operation of identifying a plurality of line groups by grouping the characters using a first threshold interval based on the width of the characters. The operation of determining the plurality of groups may include an operation of identifying a plurality of paragraph groups by grouping the plurality of line groups according to the row intervals and column intervals of each of the plurality of line groups.

[0194] For example, the operation of determining the order of the plurality of groups may include an operation of determining the order of the plurality of groups according to the recognition direction of the characters.

[0195] According to one embodiment, an electronic device as described above may include a display, a memory including one or more storage media and storing instructions, and at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a first input for executing a file representing an electronic document. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify objects within the file, groups of characters included in the electronic document, based on the first input, and to display the electronic document on the display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine a plurality of groups by grouping at least two characters positioned adjacent to each other within the electronic document using positional information of the characters represented by the objects. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine an order of the plurality of groups based on an arrangement of the plurality of groups within the electronic document. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine one or more characters using the determined order for the plurality of groups based on receiving a second input for selecting one or more characters positioned between a first character and a second character among the characters.

[0196] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display, on the display, a visual object corresponding to the second input as an overlay over the first character, the one or more characters, and the second character.

[0197] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a plurality of line groups by grouping the characters using a first threshold spacing based on the width of the characters. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a plurality of paragraph groups by grouping the plurality of line groups according to row spacings and column spacings of each of the plurality of line groups.

[0198] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine the order of the plurality of groups according to the recognition direction of the characters.

[0199] As used herein, the term "if" will be understood to mean "when, upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, "if it is determined to," or "if [the stated condition or event] is detected," will optionally be understood to mean "upon determining," or "in response to determining," "upon detecting [the stated condition or event]," or "in response to detecting [the stated condition or event]."

[0200] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0201] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0202] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0203] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0204] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In electronic devices, display; A memory including one or more storage media for storing instructions; and comprising at least one processor, comprising a processing circuit; The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Receiving a first input for executing a file comprising a first object including a first text, a second object including a second text, and a third object including a third text located between the first text and the second text; Based on the first input, display the first text, the second text, and the third text; and Based on the second input received for the first text and the second text, adjusting the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text, thereby causing a visual object to be displayed within the electronic document indicating that the first text, the third text, and the second text are selected. Electronic devices.

2. In claim 1, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Identifying a plurality of objects including the first object, the second object, and the third object; Grouping characters represented by the plurality of objects along the row direction of the electronic document, and identifying a plurality of line groups corresponding to each of the plurality of objects; Grouping the plurality of line groups by using the spacing between lines of the electronic document represented by each of the plurality of line groups, thereby identifying a plurality of paragraph groups corresponding to each of the plurality of line groups; and causing an order of the plurality of paragraph groups to be identified based on the arrangement of the plurality of paragraph groups within the electronic document, to be used for selection of text based on the second input; Electronic devices.

3. In claim 2, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: causing the above characters to be grouped using a threshold based on the width of the characters, thereby determining the plurality of line groups; Electronic devices.

4. In claim 2, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: By using whether the blank areas between the plurality of line groups are divisible along the row direction or the column direction perpendicular to the row direction, a plurality of reference points within the blank areas are determined; Causing the plurality of line groups to be grouped by using the intervals between the line groups interposed by each of the plurality of reference points and the directions of the blank areas divided by each of the plurality of reference points, Electronic devices.

5. In claim 2, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Causing to determine the order of the plurality of paragraph groups such that, while displaying the first text, the second text, and the third text of the first language, which are defined to be read in a direction from left to right, the first paragraph group, which is located on the left side of the electronic document, has a higher order than the second paragraph group, which is located on the right side of the first paragraph group. Electronic devices.

6. In claim 5, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Causing to determine the order of the plurality of paragraph groups such that the second paragraph group has a higher order than the first paragraph group while displaying the first text, the second text, and the third text in the second language, which are defined to be read in a direction from right to left, Electronic devices.

7. In claims 1 to 6, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: While displaying the above visual object, display executable objects related to the first text, the third text, and the second text; Receiving a third input for selecting one of the above executable objects; and Based on the third input, a combination of the first text, the third text, and the second text is used to cause a function associated with an executable object corresponding to the third input to be executed. Electronic devices.

8. In claims 1 to 7, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: causing the second input to be detected based on a contact point on the display that moves from the first text to the second text; Electronic devices.

9. In a non-transitory computer-readable storage medium storing instructions, the instructions, when executed by an electronic device including a display, cause the electronic device to: Receiving a first input for executing a file comprising a first object including a first text, a second object including a second text, and a third object including a third text located between the first text and the second text; Based on the first input, displaying an electronic document including the first text, the second text, and the third text; and Based on the second input received for the first text and the second text, adjusting the order of the second object and the third object according to the arrangement of the first text, the second text, and the third text, thereby causing a visual object to be displayed within the electronic document indicating that the first text, the third text, and the second text are selected. Non-transitory computer-readable storage medium.

10. In claim 9, the instructions, when executed by the electronic device, cause the electronic device to: Identifying a plurality of objects including the first object, the second object, and the third object; Grouping characters represented by the plurality of objects along the row direction of the electronic document, and identifying a plurality of line groups corresponding to each of the plurality of objects; Grouping the plurality of line groups by using the spacing between lines of the electronic document represented by each of the plurality of line groups, thereby identifying a plurality of paragraph groups corresponding to each of the plurality of line groups; and causing an order of the plurality of paragraph groups to be identified based on the arrangement of the plurality of paragraph groups within the electronic document, to be used for selection of text based on the second input; Non-transitory computer-readable storage medium.

11. In claim 10, the instructions, when executed by the electronic device, cause the electronic device to: causing the above characters to be grouped using a threshold based on the width of the characters, thereby determining the plurality of line groups; Non-transitory computer-readable storage medium.

12. In claim 10, the instructions, when executed by the electronic device, cause the electronic device to: By using whether the blank areas between the plurality of line groups are divisible along the row direction or the column direction perpendicular to the row direction, a plurality of reference points within the blank areas are determined; Causing the plurality of line groups to be grouped by using the intervals between the line groups interposed by each of the plurality of reference points and the directions of the blank areas divided by each of the plurality of reference points, Non-transitory computer-readable storage medium.

13. In claim 10, the instructions, when executed by the electronic device, cause the electronic device to: Causing to determine the order of the plurality of paragraph groups such that, while displaying the first text, the second text, and the third text of the first language, which are defined to be read in a direction from left to right, the first paragraph group, which is located on the left side of the electronic document, has a higher order than the second paragraph group, which is located on the right side of the first paragraph group. Non-transitory computer-readable storage medium.

14. In claim 13, the instructions, when executed by the electronic device, cause the electronic device to: Causing to determine the order of the plurality of paragraph groups such that the second paragraph group has a higher order than the first paragraph group while displaying the first text, the second text, and the third text in the second language, which are defined to be read in a direction from right to left, Non-transitory computer-readable storage medium.

15. In claims 9 to 14, the instructions, when executed by the electronic device, cause the electronic device to: While displaying the above visual object, display executable objects related to the first text, the third text, and the second text; Receiving a third input for selecting one of the above executable objects; and Based on the third input, a combination of the first text, the third text, and the second text is used to cause a function associated with an executable object corresponding to the third input to be executed. Non-transitory computer-readable storage medium.

Images