Automatic text recognition with layout saving

Abstract

Aspects of the subject technology include accessing, by an electronic device, a plurality of rows of textual data and textual attributes corresponding to the plurality of rows of textual data. Aspects may also include, for each respective row of the plurality of rows of the text data, selecting a plurality of rows of the text data, it is determined whether a respective row of the plurality of rows corresponds to a separate paragraph within the text data based on a first of the text attributes that corresponds to the respective row and a second of the text attributes that corresponds to a subsequent row of the plurality of rows. Aspects may also include generating output data for the plurality of rows; and performing at least one process on the lines of the text data using the generated output data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 349,031, filed on June 3, 2022, entitled “AUTOMATIC TEXT RECOGNITION WITH LAYOUT PRESERVATION,” which is incorporated herein by reference in its entirety and constitutes a part of this U.S. utility model patent application for all purposes. Technical Field

[0003] This specification generally relates to processing text data on electronic devices, including text data from image files. Background Art

[0004] Electronic devices, such as laptops, tablets, or smartphones, may be configured to access text data via a variety of formats, including images. Images may include text data that can be recognized by the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Certain features of the subject technology are set forth in the appended claims. However, for illustrative purposes, several specific implementations of the subject technology are set forth in the following figures.

[0006] Figure 1 An example network environment is illustrated according to one or more implementations.

[0007] Figure 2 Example electronic devices are depicted that may implement the subject methods and systems according to one or more implementations.

[0008] Figure 3A Depicted are example text data according to one or more implementations.

[0009] Figure 3B Depicts each row having a bounding box according to one or more specific implementations Figure 3A Sample text data for .

[0010] Figure 3C Depicting each paragraph having a bounding box according to one or more specific implementations Figure 3A Sample text data for .

[0011] Figure 3D Depicted are a method for implementing Figure 3A Example operations on sample text data.

[0012] Figure 4ADepicted are example text data with lists according to one or more implementations.

[0013] Figure 4B Depicts each row having a bounding box according to one or more specific implementations Figure 4A Sample text data for .

[0014] Figure 4C Depicting each paragraph having a bounding box according to one or more specific implementations Figure 4A Sample text data for .

[0015] Figure 4D Depicted are a method for implementing Figure 4A Example operations on sample text data.

[0016] Figure 5 Depicted is a flowchart of an example process for processing text data according to one or more implementations.

[0017] Figure 6 Depicted is a flowchart of an example process for processing text data having lists according to one or more implementations.

[0018] Figure 7 Example electronic systems are depicted that can be used to implement aspects of the present disclosure according to one or more implementations. DETAILED DESCRIPTION

[0019] The specific embodiments set forth below are intended to be descriptions of various configurations of the subject technology and are not intended to represent the only configuration by which the subject technology can be practiced. The accompanying drawings are incorporated herein and constitute a part of the specific embodiments. The specific embodiments include specific details intended to provide a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth herein, and one or more other specific implementations may be used to practice. In one or more specific implementations, structures and components are shown in block diagram form to avoid blurring the concepts of the subject technology.

[0020] The present disclosure relates to using high-order semantic understanding of text data to perform improved processing of selected text from the text data. As a non-limiting example, the high-order semantic understanding can be used to improve copy / paste operations, translation operations, dictation operations, and / or any other operations utilizing text data.

[0021] In some embodiments, the selected and / or copied text data may be formatted as columns, lists, multiple rows, etc. For example, a web page may display a news article with multiple columns. When selecting text from text data with various types of row layouts, it may be beneficial to understand the relationship between the text rows so that the semantic relationship between the rows can be preserved during operations such as copy / paste operations. For example, a recipe web page may have an "Instructions" column and an "Ingredients" column, and selecting from either column should preserve the semantic relationship between the two columns. In other words, copying the "Instructions" column and the "Ingredients" column should not connect the ingredients to the instructions, but should preserve their separation (e.g., in separate columns), as indicated by their separate columns.

[0022] Figure 1 An example network environment 100 according to one or more implementations is illustrated. However, not all of the components depicted may be used in all implementations, and one or more implementations may include additional or different components than those shown in the figures. Changes in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided. In one or more implementations, the subject method may be performed on an electronic device 102 without using the network environment 100.

[0023] The network environment 100 may include an electronic device 102 and one or more servers (e.g., server 104). A network 106 may communicatively couple (directly or indirectly) the electronic device 102 and the server 104. In one or more implementations, the network 106 may be an interconnected network that may include the Internet or devices communicatively coupled to the Internet. For purposes of illustration, the network environment 100 is shown in FIG. Figure 1 1 and 104; however, the network environment 100 may include any number of electronic devices and / or any number of servers that are communicatively coupled to each other directly or via a network 106.

[0024] The electronic device 102 may be, for example, a desktop computer; a portable computing device such as a laptop, a smart phone, a peripheral device (e.g., a digital camera, a headset), a tablet device, stand-alone video conferencing hardware, a wearable device (such as a watch, a wristband, etc.); or any other suitable device including, for example, one or more wireless interfaces, such as a WLAN radio device, a cellular radio device, a Bluetooth radio device, a Zigbee radio device, a near field communication (NFC) radio device, and / or other radio devices. In one or more specific implementations, the electronic device 102 may include a text recognition module (and / or circuit system) and one or more applications. In Figure 1In the embodiment, by way of example, the electronic device 102 is depicted as a smart phone. The electronic device 102 may be and / or may include the following Figure 7 In one or more implementations, the electronic device 102 may include a camera and a microphone, and may generate and / or provide data (e.g., images or audio) for accessing (e.g., identifying) text data for processing (e.g., via a processor or server 104).

[0025] Figure 2 An electronic device 102 is depicted that can implement the subject methods and systems according to one or more specific implementations. For illustrative purposes, this document mainly refers to Figure 1 The electronic device 102 is described Figure 2 However, this is merely illustrative and Figure 2 The features of the electronic device of can be implemented in any other electronic device (e.g., server 104) used to implement the subject technology. However, not all of the components depicted can be used in all specific implementations, and one or more specific implementations may include the same. Figure 2 Additional or different components than those shown in the drawings may be provided. Changes in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0026] The electronic device 102 may include one or more of a host processor 202, a memory 204, one or more sensors 206, and / or a communication interface 208. The host processor 202 may include appropriate logic, circuitry, and / or code capable of processing data and / or controlling the operation of the electronic device 102. In this regard, the host processor 202 may be enabled to provide control signals to various other components of the electronic device 102. The host processor 202 may also control the transfer of data between various parts of the electronic device 102. The host processor 202 may also implement an operating system or otherwise be capable of executing code to manage the operation of the electronic device 102.

[0027] The memory 204 may include appropriate logic, circuitry, and / or code that enables storage of various types of information, such as received data, generated data, code, and / or configuration information. The memory 204 may include, for example, a random access memory (RAM), a read-only memory (ROM), a flash memory, and / or a magnetic storage device. The memory 204 may store machine-readable instructions for performing the methods described herein. In one or more specific implementations, the memory 204 may store text data (e.g., as provided by the server 104). The memory 204 may also store portions of the text data for intermediate storage (e.g., in a buffer) when processing the text data.

[0028] Sensor 206 may include one or more microphones and / or cameras. A microphone may obtain an audio signal corresponding to the text data. A camera may be used to obtain an image file corresponding to the text data. For example, a camera may obtain an image of an object with text, which may be processed into text data that may be used by host processor 202 for copy / paste operations.

[0029] The communication interface 208 may include suitable logic, circuitry, and / or code to enable wired or wireless communication, such as between the electronic device 102 and the server 104. The communication interface 208 may include, for example, one or more of a Bluetooth communication interface, an NFC interface, a Zigbee communication interface, a WLAN communication interface, a USB communication interface, a cellular interface, or generally, any communication interface.

[0030] In one or more specific implementations, one or more of the host processor 202, memory 204, sensor 206, communication interface 208, and / or one or more portions thereof may be implemented in software (e.g., subroutines and code), in hardware (e.g., an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a state machine, gating logic, discrete hardware components, or any other suitable device), and / or a combination of both.

[0031] Figure 3A Describe the example text data 300 that can be selected, copied, pasted, etc. according to one or more specific implementations. The text data 300 can be retrieved from a file, stored in a data structure, identified from a photo, or identified from any other medium including text. The text data 300 can include paragraphs 302 to 310, which represent discrete parts (e.g., titles, subtitles, line sets, etc.) of the text data 300 separated by line spacing (e.g., line feed characters) between each other. Paragraphs 302 to 310 can include one or more lines. For example, paragraph 302 includes line 312, paragraph 304 includes line 314 to line 330, paragraph 306 includes line 332 to line 360, paragraph 308 includes line 362, and paragraph 310 includes line 364 to line 384.

[0032] Although, for example, line 304 of paragraph Figure 3A300 as having line breaks between them, a semantic-based understanding of text data 300 allows the subject system to disambiguate between line breaks inherent in text data 300 (such as a line break at the end of paragraph 304) and line breaks in text data 300 that are a result of formatting of text data 300 (such as a line break at the end of each of lines 314 to 328). Thus, in the subject system, line breaks inherent in text data 300 can be preserved, while line breaks resulting from the specific formatting of text data 300 can be discarded.

[0033] Figure 3B Each row 312 to row 384 is depicted as having a bounding box according to one or more specific implementations. Figure 3A Each of lines 312 to 384 may be determined to correspond to one of paragraphs 302 to 310 based on semantic information and / or geometric information corresponding to each of lines 312 to 384.

[0034] Semantic information may include, for example, punctuation, symbols, capitalization, word count, part-of-speech tags (e.g., nouns, verbs, adjectives, etc. as determined by a natural language processing part-of-speech tagging algorithm), and / or any other information related to the semantics of the text data. For example, row 328 and row 330 may correspond to the same paragraph because row 328 does not end with a period, while row 330 and row 332 may correspond to different paragraphs because row 330 ends with a period. Row 330 and row 332 may also correspond to different paragraphs because row 332 begins with a capital letter. For another example, if a row ends with a preposition, the row may be merged with the next row because rows do not usually end with a preposition. However, rows ending with a period, rows beginning with a capital letter, and / or rows ending with a preposition cannot individually and decisively identify different paragraphs.

[0035] The geometric information may include a row start position, a row height, a row spatial orientation, a row length, a row spacing, and / or any other information related to the geometric structure of the row. In one or more specific implementations, the row surrounded by the bounding box may be used to train the machine learning model to output a bounding box corresponding to the row used as input. The bounding box may be displayed or not displayed to the user. The bounding box may be used to reflect the geometric information of the row. For example, rows 314 to 330 may be determined to belong to the same paragraph because they are substantially the same size (e.g., height), have substantially the same spatial orientation, and have substantially the same starting position. Although row 312 has the same starting position, the row has a different size (e.g., height) relative to adjacent rows, which may indicate that row 312 is a header. Although rows 332 to 360 have the same starting position, they are separated from row 330 by a line spacing.

[0036] Figure 3CDepicted is a diagram having a bounding box around each of paragraphs 302-310 according to one or more implementations. Figure 3A Example text data 300. Pairs of sequential lines 312 to 384 may be analyzed to determine whether the pairs correspond to separate paragraphs. In one or more implementations, when the analysis determines that lines of text data 300 (e.g., separated by space characters) correspond to the same paragraph, the lines may be merged, and when the analysis determines that the lines correspond to the end of a paragraph, a line break may be inserted (or maintained). In one or more implementations, the text data 300 may be analyzed, and corresponding metadata may be generated to indicate which lines 312 to 384 belong to the same paragraph 302 to 310 and / or to indicate when a line corresponds to the end of a paragraph (and / or the beginning of a paragraph). In one or more implementations, the bounding boxes of lines 312 to 384 may be merged based on the determined paragraph separation that produces a bounding box corresponding to paragraphs 302 to 310. The analysis process may occur multiple times, merging lines into paragraphs until the lines are no longer merged. About Figure 5 and Figure 6 The analysis process of lines 312 to 384 is discussed in more detail.

[0037] Figure 3D Depicted are a method for implementing Figure 3A The example operations (e.g., copy operation 390 and paste operation 394) of the example text data 300 of FIG. 300 are shown in FIG. 301 . After analyzing rows 312 to 384, output data indicating which rows of rows 312 to 384 of the text data 300 correspond to separate paragraphs can be generated for rows 312 to 384. The output data can be used to create a data structure (e.g., a buffer) with the text data 300, which is divided into paragraphs 302 to 310 according to the output data. In one or more specific implementations, the output data can be used to modify the text data 300 when performing operations on the text data 300.

[0038] For example, the operation may include a copy operation 390 and a paste operation 394. The user may select a portion of the text data 300, such as paragraph 302 and paragraph 304, as shown in the selection indicator 392. The user may select by touching, clicking, or generating any other input using an electronic device (e.g., electronic device 102). The user may initiate the copy operation 390 by, for example, tapping, clicking, or generating any other input on the selection indicator 392 using an electronic device and selecting the copy operation 390. When the copy operation 390 is initiated, the electronic device may copy the text data selected by the selection indicator 392 from the data structure to the clipboard so that it is formatted semantically (e.g., by paragraph), rather than formatted when the text is displayed (e.g., each line is considered a separate paragraph). In one or more specific implementations, when the copy operation 390 is initiated, the electronic device may copy the text data formatted as shown and the corresponding output data, and apply the output data so that when the operation is completed, the text data selected by the selection indicator 392 is formatted semantically (e.g., the selected text data is in the clipboard with semantically based formatting).

[0039] To perform a paste operation 394, a user may change to an application 386 having an input box 388, and tap, click, or generate any other input on the input box 388 using an electronic device, and select a paste operation 394. In a typical paste operation, the text data selected by the selection indicator 392 may appear in the input box 388 so that each line 312 to line 384 is formatted to be presented to the user (e.g., treated as a separate paragraph). In a paste operation 394 corresponding to the analysis of the subject technology, the text data selected by the selection indicator 392 may appear in the input box 388 so that the text selected by the selection indicator 392 is semantically formatted (e.g., by paragraph). For example, paragraph 302 is separated from paragraph 304, and each line of paragraph 304 is merged into paragraph 304 (e.g., and each line is separated by a line spacing that can be inserted as needed by the subject system), so that a new line character is placed at the end of paragraph 304.

[0040] Figure 4ADepicted is an example text data 400 with a list 420 according to one or more implementations. The text data 400 may be retrieved from a file, stored in a data structure, identified from a photograph, or identified from any other medium including text. The text data 400 may include paragraphs 402 to 418, which represent discrete portions (e.g., headings, subheadings, sets of lines, lists, etc.) of the text data 400 separated by corresponding line spacing (e.g., line feed characters) between each other. Paragraphs 402 to 418 may include one or more lines. For example, paragraph 402 includes line 422, paragraph 404 includes lines 424 to 434, paragraph 406 includes lines 436 to 442, paragraph 408 includes lines 444 to 446, paragraph 410 includes lines 448 to 456, paragraph 412 includes lines 458 to 462, paragraph 414 includes lines 464 to 482, paragraph 416 includes line 484, and paragraph 418 includes lines 486 to 488.

[0041] Figure 4B Depicts each row having a bounding box according to one or more specific implementations Figure 4A Each of lines 422 to 488 may be determined to correspond to one of paragraphs 402 to 418 based on semantic information and / or geometric information corresponding to each of lines 422 to 488.

[0042] Semantic information may include punctuation, symbols, capitalization, word count, part-of-speech tags, and / or any other information related to the semantics of the text data. For example, row 424 and row 426 may correspond to the same paragraph because row 424 does not end with a period, while row 434 and row 436 may correspond to different paragraphs because row 434 ends with a period. Row 434 and row 436 may also correspond to different paragraphs because row 436 begins with a capital letter. In one or more specific implementations, the semantic information of rows 436 to 482 may indicate that these rows belong to a list. For example, row 436, row 444, row 448, and row 464 begin with a list item indicator (e.g., 1, 2, 3). The sequential number list item indicator may indicate that paragraph 406, paragraph 410, and paragraph 414 belong to the same list. Although the list item indicator corresponding to paragraph 408 is sequential, the list item indicator is alphabetical and, therefore, may indicate that the list item indicator is part of a separate list (eg, a sub-list).

[0043] The geometric information may include a row start position, a row height, a row spatial orientation, a row length, a row spacing, and / or any other information related to the geometric structure of the row. In one or more specific implementations, a machine learning model may be trained using rows and corresponding bounding boxes to output a bounding box corresponding to the row used as input. The bounding box may be displayed or not displayed to the user. The bounding box may be used to reflect the geometric information of the row. For example, rows 424 to 434 may belong to the same paragraph 404 because they are the same size (e.g., length and / or height), have the same spatial orientation, and have the same starting position. Although row 422 has the same starting position, the row has a different size (e.g., height) relative to adjacent rows, which may indicate that row 422 is a header. Although rows 486 to 488 have the same starting position, they are separated from rows 424 to 434 by rows 436 to 482 that do not have the same starting position. The geometric information of rows 436 to 482 may indicate that they belong to a list. For example, lines 436 to 442, lines 448 to 482 all have the same second line starting position (e.g., an indented line starting position). The second line starting position may indicate that paragraphs 406, 410, 412, and 414 belong to the same list. Paragraph 408 has a third line starting position (e.g., a double indented line starting position), which may indicate that it is part of a separate list (e.g., a sublist).

[0044] Figure 4C Depicted is a diagram with a bounding box surrounding each of paragraphs 402 through 418 according to one or more implementations. Figure 4A Example text data 400. Pairs of lines 422 through 488 may be analyzed to determine whether the pairs correspond to separate paragraphs. In one or more implementations, when the analysis determines that lines of text data 400 (e.g., separated by space characters) correspond to the same paragraph, the lines may be merged, and when the analysis determines that the lines correspond to the end of a paragraph, a line break may be inserted (or maintained). In one or more implementations, text data 400 may be analyzed, and corresponding metadata may be generated to indicate which lines 422 through 488 belong to the same paragraph 402 through 418. In one or more implementations, the bounding boxes of lines 422 through 488 may be merged based on the determined paragraph separation that produces bounding boxes corresponding to paragraphs 402 through 418. Figure 5 and Figure 6 The analysis process of lines 422 to 488 is discussed in more detail.

[0045] Figure 4D Depicted are a method for implementing Figure 4A494 and paste operation 498). After analyzing rows 422 to 488, output data indicating which rows of rows 422 to 488 of text data 400 correspond to individual paragraphs and / or lists can be generated for rows 422 to 488. The output data can be used to create a data structure (e.g., a buffer) with text data 400, which is divided into paragraphs 402 to 418 and / or lists (e.g., list 420) according to the output data. In one or more specific implementations, the output data can be used to modify text data 400 when performing operations on text data 400.

[0046] The operation may include a copy operation 494 and a paste operation 498. The user may select a portion of the text data 400, such as paragraphs 406 to 412 and a portion of paragraph 414, as shown in the selection indicator 496. The user may select by touching, clicking, or generating any other input using an electronic device (e.g., electronic device 102). The user may initiate a copy operation 494 by, for example, tapping, clicking, or generating any other input on the selection indicator 496 using an electronic device and selecting the copy operation 494. When the copy operation 494 is initiated, the electronic device may copy the text data selected by the selection indicator 496 from the data structure to the clipboard so that it is formatted semantically (e.g., by paragraphs and lists), rather than formatted as shown (e.g., each line is considered a separate paragraph). In one or more specific implementations, when the copy operation 494 is initiated, the electronic device may copy the text data formatted as shown and the corresponding output data, and apply the output data so that when the operation is completed, the text data selected by the selection indicator 496 is formatted semantically (e.g., the selected text data is in the clipboard with semantically based formatting).

[0047] To perform a paste operation 498, a user may change to an application 490 having an input box 492, and tap, click, or generate any other input on the input box 492 using an electronic device, and select a paste operation 498. In a typical paste operation, the text data selected by the selection indicator 496 may appear in the input box 492 so that each line 422 to line 488 is formatted to be presented to the user (e.g., treated as a separate paragraph). In a paste operation 498 corresponding to the analysis of the subject technology, the text data selected by the selection indicator 496 may appear in the input box 492 so that the text selected by the selection indicator 496 is semantically formatted (e.g., by paragraph and list). For example, paragraph 406 is separated from paragraph 408, and each line of paragraph 406 is merged into paragraph 406 so that a new line character is placed at the end of paragraph 406. In one or more implementations, text data selected by selection indicator 496 can be pasted in a format (eg, rich text format) such that lists (eg, list 420 ) and sub-lists are formatted in a list format.

[0048] Figure 5 A flowchart of an example process 500 for processing text data according to one or more specific implementations is depicted. For illustrative purposes, this document mainly refers to Figure 1 The process 500 is described with reference to the electronic device 102 of the present invention. However, the process 500 is not limited to the electronic device 102, and one or more blocks of the process 500 may be performed by one or more other components of the electronic device 102 and / or other appropriate devices. Further for the purpose of illustration, the blocks of the process 500 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 500 may occur in parallel. In addition, the blocks of the process 500 do not have to be executed in the order shown, and / or one or more blocks of the process 500 do not have to be executed and / or may be replaced by other operations. In one or more specific implementations, the application stored on the electronic device 102 executes the process 500 by calling an API provided by the operating system of the electronic device 102. In one or more specific implementations, the operating system of the electronic device 102 executes the process 500 by processing the API call provided by the application stored on the electronic device 102. In one or more specific implementations, the application stored on the electronic device 102 executes the process 500 completely without making any API calls to the operating system of the electronic device 102.

[0049] At box 502, multiple lines (e.g., lines 312 to 384) of text data (e.g., text data 300) can be accessed. An electronic device (e.g., electronic device 102) can access multiple lines and / or corresponding text attributes from a data structure (such as a file). In one or more specific implementations, accessing multiple lines may include: receiving a file; identifying text data; and accessing the identified text data. For example, an electronic device may receive an image of an object with text, perform text recognition on the image (e.g., via an image processing algorithm), and access text data from the image with the identified text. For another example, a server (e.g., server 104) may receive an image of an object with text and perform text recognition on the image (e.g., via an image processing algorithm), and the electronic device may access the text data via the server.

[0050] Text attributes corresponding to multiple lines of text data may also be accessed. The text attributes of the text data may include semantic information and / or geometric information. The semantic information may include punctuation, symbols, capitalization, word count, part-of-speech tags, and / or any other information related to the semantics of the text data. The geometric information may include line start position, line height, line space orientation, line length, line spacing, and / or any other information related to the geometric structure of the line as displayed / formatted in a file, image, etc. In one or more specific implementations, accessing the text attributes may include: receiving an image including multiple lines of text data; and generating one or more bounding boxes associated with one or more lines of the multiple lines of text data (e.g., via an image processing algorithm).

[0051] In one or more specific implementations, the electronic device can determine the language corresponding to the text data so that process 500 can be performed based on the reading order corresponding to the language. For example, the electronic device can use a natural language processing model (e.g., a language detection model) to determine that the language of the text data is Traditional Chinese, and modify process 500 so that the text line is analyzed from right to left (because the line is vertical) instead of from top to bottom (if the line is horizontal).

[0052] At box 504, it can be determined whether the corresponding line and the subsequent line correspond to a separate paragraph in the text data. This determination can be based on a first text attribute corresponding to the corresponding line in the text attribute and a second text attribute corresponding to the subsequent line in the text attribute. This determination can be performed by a set of heuristics, a trained machine learning model, or any appropriate method to determine whether these lines belong to a separate paragraph. If two lines of text (e.g., the corresponding line and the subsequent line) belong to a separate paragraph, a line spacing (e.g., a line break) can be inserted between the two lines of text. If the two lines of text belong to the same paragraph, a space (e.g., a space character) can be inserted between the two lines of text (or the space can replace an existing line break).

[0053] Heuristics and / or signals that can be used to make such determinations may include, but are not limited to, language-specific heuristics, grouping markers (e.g., bounding boxes) applied to lines, spatial orientation of identified groups within the selected text, natural language processing results, and the like. For example, a natural language processing algorithm may perform part-of-speech tagging on at least the first and last words of each line, and the heuristics may include rules for parts of speech that may be merged. Such rules may include: if the first line ends with a preposition, merge the two lines. For another example, a computer vision algorithm may apply a bounding box to each line and the heuristics may include rules such that lines with the same starting position, height, and / or orientation may be merged.

[0054] The machine learning model may be trained using training data that includes text lines with labels indicating text attributes and corresponding determinations (e.g., probabilities) of whether pairs of text lines have matching text attributes (e.g., semantic information and / or geometric information). Thus, the input to the machine learning model may include line pairs and the text attributes of each line, and the output to the machine learning model may include a determination of whether the line pairs have matching attributes. For example, each line pair of text lines may be labeled as having a particular set of text attributes via the output of the machine learning model (e.g., in metadata), and the lines may be merged into paragraphs or divided into paragraphs based on their labels.

[0055] In one or more specific implementations, the machine learning model may also or alternatively be trained using training data that includes text lines with labels indicating text attributes and corresponding determinations of whether pairs of text lines correspond to the same paragraph. Thus, the input to the machine learning model may include line pairs and the text attributes of each line, and the output to the machine learning model may include the likelihood / probability of whether the line pairs correspond to the same paragraph and / or different paragraphs. For example, the selected text may be segmented into groups, spatial information associated with the identified groups may be collected, and natural language processing may be performed on the selected text (according to the rules of the language corresponding to the text), and the spatial information and natural language processing results may be used as input to the trained machine learning model to determine whether input lines of text are likely to belong to separate paragraphs.

[0056] At box 506, output data may be generated for the multiple lines. The output data may indicate which lines of the multiple lines of text data correspond to separate paragraphs. For example, the output data may be instructions for merging lines or metadata identifying lines as belonging to the same paragraph. The output data may be generated by one or more machine learning models, heuristics, or any other appropriate method for determining whether spaces (e.g., line breaks or space characters) should be inserted between two lines of text (e.g., corresponding lines and subsequent lines). Additionally or alternatively, the output data may include text lines corresponding to instructions for which lines of the multiple lines of text data correspond to separate paragraphs. In one or more specific implementations, the output data includes text lines with line breaks added or removed as appropriate to place lines in separate paragraphs and text lines with space characters added or removed as appropriate to place lines in the same paragraph. In one or more specific implementations, the output data may be generated in an incremental manner so that more data is added to the output data when process 500 iterates through each line of the multiple lines of the text data.

[0057] At box 508, determine whether there are more rows to be analyzed. In one or more specific implementations, each row of the text data can be analyzed. For example, each row of the text data can be analyzed when the image is received so that the user can use the text data in the image after opening the image. In one or more specific implementations, only the selection of the rows of the text data can be analyzed. For example, the selection of the rows of the text data corresponding to a portion of the image can be analyzed when the image is loaded, rendered, etc., so as to reduce the computational burden on the electronic device. If there are more rows to be analyzed, process 500 can return to box 504. If there are no more rows to be analyzed, the process can continue to box 510.

[0058] At box 510, at least one process may be performed on the multiple lines of the text data using the generated output data. In one or more specific implementations, the multiple lines of the text data may be modified based on the output data and copied to a clipboard. For example, the output data may include metadata describing which lines belong to the same paragraph and / or different paragraphs, and the multiple lines of the text data may be modified based on the metadata by adding or removing line breaks between lines as needed to place lines in the same paragraph.

[0059] In one or more specific implementations, the process can be a copy / paste operation. For example, a user can select one or more lines of text or a portion thereof, and perform a copy operation (e.g., copy operation 390) to copy the selection to a clipboard. The selection can have line breaks inserted or removed as needed to place the lines in the selection in a separate paragraph as shown in the text data. The selection can also have character spaces inserted at the end of one or more lines as needed to prevent words from two separate lines from being merged together. The selection can also or alternatively have metadata indicating that the lines in the selection belong to separate paragraphs. When a paste operation (e.g., paste operation 394) is performed, the selection can be pasted so that the selection is arranged in a paragraph as shown in the text data (e.g., as laid out in an image).

[0060] In one or more specific implementations, the output data may be provided to an application or system process. The application or system process may include a file. For example, the output data may be written to a text file. The application or system process may also or alternatively include a data structure. For example, the output data may be written to a buffer in a memory. The application or system process may also or alternatively include a translation process. For example, a machine learning model trained to translate a first language into a second language may receive output data including text data in the first language as input and output text data in the second language. The application or system process may also or alternatively include a dictation process. For example, the output data may correspond to text data in an audio format and be used as input to a machine learning model trained to convert speech to text. The application or system process may also or alternatively include a narration process. For example, the output data may be used as input to a machine learning model that is trained to convert text into an audio format based on the output data, where the audio reads the text as consecutive sentences of lines corresponding to the same paragraph. The application or system process may also or alternatively include a virtual assistant process. For example, the output data may be used as a request to a virtual assistant that processes the request. In one or more specific implementations, these processes can be combined with each other. For example, the narration process can receive output data for narration and pass it to the audio generation process to generate an audio file for narrating text data corresponding to the output data.

[0061] Figure 6 A flowchart of an example process 600 for processing text data having a list according to one or more specific implementations is depicted. For illustrative purposes, this document mainly refers to Figure 1The process 600 is described with reference to the electronic device 102 of the present invention. However, the process 600 is not limited to the electronic device 102, and one or more blocks of the process 600 may be performed by one or more other components of the electronic device 102 and / or other appropriate devices. Further for the purpose of illustration, the blocks of the process 600 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 600 may occur in parallel. In addition, the blocks of the process 600 do not have to be executed in the order shown, and / or one or more blocks of the process 600 do not have to be executed and / or may be replaced by other operations. In one or more specific implementations, the application stored on the electronic device 102 executes the process 600 by calling an API provided by the operating system of the electronic device 102. In one or more specific implementations, the operating system of the electronic device 102 executes the process 600 by processing the API call provided by the application stored on the electronic device 102. In one or more specific implementations, the application stored on the electronic device 102 executes the process 600 completely without making any API call to the operating system of the electronic device 102.

[0062] At box 602, multiple lines (e.g., lines 422 to 488) of text data (e.g., text data 400) can be accessed. An electronic device (e.g., electronic device 102) can access multiple lines and / or corresponding text attributes from a data structure (such as a file). In one or more specific implementations, accessing multiple lines may include: receiving a file; identifying text data; and accessing the identified text data. For example, an electronic device may receive an image of an object with text, perform text recognition on the image (e.g., via an image processing algorithm), and access text data from the image with the identified text. For another example, a server (e.g., server 104) may receive an image of an object with text and perform text recognition on the image (e.g., via an image processing algorithm), and the electronic device may access the text data via the server.

[0063] At box 604, a first list item row and a second list item row (e.g., row 436 and row 448) from the plurality of rows (e.g., rows 422 through 488) are identified. List item rows are rows that begin with a list item indicator. For example, a list item row for an unenumerated list begins with a list item indicator that is a black dot, a dash, an asterisk, or any other symbol that is common between each list item row within the list. For another example, a list item row for an enumerated list begins with a list item indicator that is a number, a letter, or any other sequential symbol that is common between each list item row within the list (e.g., row 436 and row 448 begin with a number).

[0064] At box 606, a list entry is generated based on the first list item row and each corresponding line located between the first list item row and the second list item row. The first list item row may include multiple lines, and thus the first list item row and the second list item row may be separated by a number of text lines (e.g., lines 438 to 446 between lines 436 and 448). The text lines between the first list item row and the second list item row may be part of a list (e.g., lines 438 to 442 between lines 436 and 448), part of a separate list (e.g., a sublist at lines 444 to 446), or not part of a list or a separate list. In one or more specific implementations, there may be no text lines between the first list item row and the second list item row. In this case, a list item entry may be generated for the first list item row and the second list item row, and the list item entry may continue to the next set of list item rows (e.g., by skipping box 608).

[0065] At block 608, a determination may be made as to whether the corresponding line and the subsequent line correspond to separate paragraphs within the list entry. The determination may be based on a first text attribute in the text attributes that corresponds to the corresponding line and a second text attribute in the text attributes that corresponds to the subsequent line. The determination may be made by a set of heuristics, a trained machine learning model, or any suitable method to determine whether the lines belong to separate paragraphs as described above with respect to process 500. If the two lines of text (e.g., the corresponding line and the subsequent line) belong to separate paragraphs, line spacing (e.g., a line break) may be inserted between the two lines of text. If the two lines of text belong to the same paragraph, a space (e.g., a space character) may be inserted between the two lines of text (or the space may replace an existing line break).

[0066] At box 610, output data may be generated for the multiple lines. The output data may indicate which lines of the multiple lines of text data correspond to separate paragraphs and / or separate lists. For example, the output data may be instructions for merging lines or metadata identifying lines as belonging to the same paragraph and / or list. The output data may be generated by one or more machine learning models, heuristics, or any other appropriate method for determining whether spaces (e.g., line breaks or space characters) should be inserted between two lines of text (e.g., corresponding lines and subsequent lines). Additionally or alternatively, the output data may include text lines corresponding to indications of which lines of the multiple lines of text data correspond to separate paragraphs and / or lists. In one or more specific implementations, the output data includes text lines with line breaks added or removed as appropriate to place lines in separate paragraphs and / or lists and text lines with space characters added or removed as appropriate to place lines in the same paragraph. In one or more specific implementations, the output data may be generated in an incremental manner so that more data is added to the output data when process 600 iterates through each of the multiple lines of the text data.

[0067] At block 612, a determination is made as to whether there are more rows to be analyzed between the first list item row and the second list item row. In one or more implementations, one or more rows between the first list item row and the second list item row may correspond to a separate list (e.g., a sublist), in which case process 600 is recursively performed on the first list item row and the second list item row of the sublist. In one or more implementations, there may not be a second list item row (e.g., because process 600 has reached the last item on the list). In this case, the remainder of the text data may be analyzed according to process 500 or process 600, as appropriate.

[0068] At box 614, at least one process may be performed on the multiple lines of the text data using the generated output data. In one or more specific implementations, the multiple lines of the text data may be modified based on the output data and copied to a clipboard. For example, the output data may include metadata describing which lines belong to the same paragraph and / or list, and the multiple lines of the text data may be modified based on the metadata by adding or removing line breaks or space characters between lines as needed to place lines in the same paragraph and / or list.

[0069] In one or more specific implementations, the process can be a copy / paste operation. For example, a user can select one or more lines of text or a portion thereof and perform a copy operation (e.g., copy operation 390) to copy the selection to a clipboard. The selection can have line breaks inserted or removed as needed to place the lines within the selection in a separate paragraph as shown in the text data. The selection can also or alternatively have metadata indicating that the lines within the selection belong to a separate paragraph. When a paste operation (e.g., paste operation 394) is performed, the selection can be pasted so that the selection is arranged in an indented paragraph and list as shown in the text data (e.g., as laid out in the image).

[0070] In one or more specific implementations, the output data may be provided to an application or system process. The application or system process may include a file. For example, the output data may be written to a text file, wherein list formatting is applied to the list items of the output data. The application or system process may also or alternatively include a data structure. For example, the output data may be written to a buffer in a memory. The application or system process may also or alternatively include a translation process. For example, a machine learning model trained to translate a first language into a second language may receive output data including text data in the first language as input, and output text data in the second language, wherein the list item row indicator may also be translated into the second language. The application or system process may also or alternatively include a dictation process. For example, the output data may correspond to text data in an audio format and be used as input to a machine learning model trained to convert speech to text, wherein each list item in the list is read by pausing between each list item rather than as an entire paragraph. The application or system process may also or alternatively include a narration process. For example, the output data can be used as input to a machine learning model that is trained to convert text into an audio format based on the output data, where the audio reads the text as a list, pausing between each list item rather than reading each item continuously. The application or system process may also or alternatively include a virtual assistant process. For example, the output data can be used as a request to a virtual assistant, which processes the request. In one or more specific implementations, these processes can be combined with each other. For example, a narration process can receive output data for narration and pass it to an audio generation process to generate an audio file for narrating text data corresponding to the output data.

[0071] As described above, one aspect of the present technology is to collect and use data available from specific and legitimate sources for processing text data. The present disclosure contemplates that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data may include demographic data, location-based data, online identifiers, phone numbers, email addresses, home addresses, images, videos, audio data, data or records related to the user's health or fitness level (e.g., vital signs measurements, medication information, exercise information), date of birth, or any other personal information.

[0072] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, personal information data can be used to process text data. Therefore, the use of such personal information data can facilitate transaction processing (e.g., online transaction processing). In addition, the present disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used according to user preferences to provide insights into their overall health status, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0073] The present disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities will be expected to implement and consistently apply privacy practices that are generally recognized as meeting or exceeding the requirements of the industry or government that maintain user privacy. Such information about the use of personal data should be accessible to users in large quantities and conveniently, and should be updated as the collection and / or use of data changes. The user's personal information should be collected only for legal use. In addition, such collection / sharing should only occur after receiving user consent or other legal basis specified in applicable law. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that other entities with access to personal information data comply with their privacy policies and procedures. In addition, such entities may subject themselves to third-party assessments to demonstrate that they comply with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted for specific types of personal information data collected and / or accessed, and applied to applicable laws and standards, including specific considerations that are exclusive to jurisdictions that can be used to impose higher standards. For example, in the United States, collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly.

[0074] Regardless of the foregoing, the present disclosure also contemplates specific implementations in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates hardware elements and / or software elements to prevent or block access to such personal information data. For example, with respect to processing text data, the technology of the present invention may be configured to allow a user to choose to "opt in" or "opt out" to participate in the collection of personal information data at any time during or after registration for a service. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then be reminded again just before the personal information data is accessed by the application.

[0075] In addition, it is the intention of the present disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting data collection and data deletion. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be promoted when appropriate by removing identifiers, controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods such as differential privacy.

[0076] Therefore, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed implementations, the present disclosure also contemplates that various implementations may be implemented without access to such personal information data. That is, various implementations of the present technology will not fail to function properly due to the lack of all or part of such personal information data.

[0077] Figure 7 An example electronic system 700 is depicted that can be used to implement aspects of the present disclosure according to one or more implementations. The electronic system 700 can be used to generate a reference Figure 1 9 and / or may be a part thereof, including but not limited to laptop computers, tablet computers, smart phones, and wearable devices (e.g., smart watches, fitness bands). Electronic system 700 may include various types of computer-readable media and interfaces for various other types of computer-readable media. Electronic system 700 includes one or more processing units 714, persistent storage device 702, system memory 704 (and / or buffer), input device interface 706, output device interface 708, bus 710, ROM 712, one or more processing units 714, one or more network interfaces 716, and / or subsets and variations thereof.

[0078] The bus 710 generally represents all system buses, peripheral busses, and chipset buses that communicatively connect the various internal devices of the electronic system 700. In one or more specific implementations, the bus 710 communicatively connects one or more processing units 714 with the ROM 712, the system memory 704, and the persistent storage device 702. The one or more processing units 714 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed in the present subject matter. In different specific implementations, the one or more processing units 714 can be a single processor or a multi-core processor.

[0079] ROM 712 stores static data and instructions required by one or more processing units 714 and other modules of electronic system 700. On the other hand, persistent storage device 702 can be a read-write memory device. Persistent storage device 702 can be a non-volatile memory unit that stores instructions and data even when electronic system 700 is turned off. In one or more specific implementations, a mass storage device (such as a magnetic disk or optical disk and its corresponding disk drive) can be used as persistent storage device 702.

[0080] In one or more implementations, a removable storage device (such as a floppy disk, a flash drive, and its corresponding disk drive) may be used as the persistent storage device 702. Like the persistent storage device 702, the system memory 704 may be a read-write memory device. However, unlike the persistent storage device 702, the system memory 704 may be a volatile read-write memory, such as RAM. The system memory 704 may store any of the instructions and data that one or more processing units 714 may need at runtime. In one or more implementations, the processes disclosed in the subject matter are stored in the system memory 704, the persistent storage device 702, and / or the ROM 712. The one or more processing units 714 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes of one or more implementations.

[0081] The bus 710 is also connected to an input device interface 706 and an output device interface 708. The input device interface 706 enables a user to transmit information and select commands to the electronic system 700. Input devices that can be used with the input device interface 706 may include, for example, an alphanumeric keyboard, a touch screen, and a pointing device (also referred to as a "cursor control device"). The output device interface 708 may, for example, enable the display of images generated by the electronic system 700. Output devices that can be used with the output device interface 708 may include, for example, a printer and a display device, such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flexible display, a flat panel display, a solid state display, a projector, or any other device for outputting information.

[0082] One or more implementations may include a device that acts as both an input device and an output device, such as a touch screen. In these implementations, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user may be received in any form, including acoustic, voice, or tactile input.

[0083] Finally, if Figure 7 As shown, bus 710 also couples electronic system 700 to one or more networks and / or to one or more network nodes via one or more network interfaces 716. In this manner, electronic system 700 may be part of a computer network such as a LAN, a wide area network ("WAN"), an intranet, or may be part of a network of networks such as the Internet. Any or all components of electronic system 700 may be used in conjunction with the subject disclosure.

[0084] Implementations within the scope of the present disclosure may be implemented in part or in whole using a tangible computer-readable storage medium (or multiple tangible computer-readable storage media of one or more types) encoding one or more instructions. Tangible computer-readable storage media may also be non-transitory in nature.

[0085] Computer-readable storage media can be any storage media that can be read, written, or otherwise accessed by a general or special computing device, including any processing electronics and / or processing circuitry capable of executing instructions. For example, without limitation, computer-readable media can include any volatile semiconductor memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. Computer-readable media can also include any non-volatile semiconductor memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, flash memory, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, racetrack memory, FJG, and Millipede memory.

[0086] In addition, the computer-readable storage medium may include any non-semiconductor memory, such as optical disk storage, magnetic disk storage, magnetic tape, other magnetic storage devices, or any other medium capable of storing one or more instructions. In one or more specific implementations, the tangible computer-readable storage medium may be directly coupled to the computing device, while in other specific implementations, the tangible computer-readable storage medium may be indirectly coupled to the computing device, for example, via one or more wired connections, one or more wireless connections, or any combination thereof.

[0087] Instructions can be directly executable, or can be used to develop executable instructions. For example, instructions can be implemented as executable or non-executable machine code, or can be implemented as high-level language instructions that can be compiled to produce executable or non-executable machine code. In addition, instructions can also be implemented as data, or can include data. Computer executable instructions can also be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, etc. As recognized by those skilled in the art, the details including but not limited to the number, structure, sequence and organization of instructions can be significantly different without changing the underlying logic, function, processing and output.

[0088] Although the above discussion mainly involves microprocessors or multi-core processors that execute software, one or more specific implementations are performed by one or more integrated circuits (such as ASICs or FPGAs). In one or more specific implementations, such integrated circuits execute instructions stored on the circuit itself.

[0089] Those skilled in the art will recognize that the various illustrative blocks, modules, elements, components, methods and algorithms described herein can be implemented as electronic hardware, computer software or a combination of the two. In order to illustrate this interchangeability of hardware and software, various illustrative blocks, modules, elements, components, methods and algorithms have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. The technician can implement the described functionality in different ways for each specific application. Various components and blocks can be arranged in different ways (e.g., arranged in different orders, or divided in different ways) without departing from the scope of the subject technology.

[0090] It should be understood that the specific order or hierarchy of the frames in the process disclosed by the present invention is an illustration of the example method. Based on the design preference requirements, it should be understood that the specific order or hierarchy of the frames in the process can be rearranged or all the illustrated frames are executed. Any frame in these frames can be executed simultaneously. In one or more specific implementations, multitasking and parallel processing may be advantageous. In addition, the division of each system component in the above-mentioned specific implementation should not be understood as requiring such division in all specific implementations, and it should be understood that program components and systems can be generally integrated together in a single software product or encapsulated in multiple software products.

[0091] As used in this specification and any claims of this patent application, the terms "base station", "receiver", "computer", "server", "processor" and "memory" all refer to electronic devices or other technical devices. These terms exclude people or groups of people. For the purpose of this specification, the term "display" or "displaying" means displaying on an electronic device.

[0092] As used herein, the phrase "at least one of" following a list of items, any of which is separated by the terms "and" or "or", modifies the list as a whole, rather than each member (i.e., each item) of the list. The phrase "at least one of" does not require selection of at least one of each item listed; rather, the phrase allows for a meaning that includes at least one of any one item and / or at least one of any combination of items and / or at least one of each item. By way of example, the phrases "at least one of A, B, and C" or "at least one of A, B, or C" each refer to only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0093] The predicate words "configured to," "operable to," and "programmed to" do not imply any specific tangible or intangible modification of a subject matter but are intended to be used interchangeably. In one or more specific implementations, a processor configured to monitor and control an operation or component may also mean that the processor is programmed to monitor and control the operation or that the processor is operable to monitor and control the operation. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

[0094] Phrases such as aspect, this aspect, another aspect, some aspects, one or more aspects, a specific implementation, this specific implementation, another specific implementation, one or more specific implementations, one or more specific implementations, implementation, this implementation, another implementation, one or more specific implementations, one or more implementations, configuration, this configuration, another configuration, some configurations, one or more configurations, subject technology, disclosure, the present disclosure, other variations thereof, and the like are for convenience and do not mean that disclosures involving such phrases are essential to the subject technology, nor do they mean that such disclosures apply to all configurations of the subject technology. Disclosures involving such one or more phrases may apply to all configurations or one or more configurations. Disclosures involving such one or more phrases may provide one or more examples. Phrases such as aspect or some aspects may refer to one or more aspects and vice versa, and this applies similarly to the other aforementioned phrases.

[0095] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" or as an "example" is not necessarily to be construed as preferred or advantageous over other implementations. In addition, to the extent that the terms "including," "having," and the like are used in the specification or claims, such terms are intended to be inclusive in a manner similar to how the term "comprising" is interpreted when used as a transitional word in a claim.

[0096] All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be made available to the public regardless of whether the disclosure is explicitly stated in the claims. Pursuant to 35 U.S.C. §112, sixth paragraph, no claim element shall be construed unless the element is expressly stated using the phrase “means for” or, in the case of a method claim, the element is stated using the phrase “step for”.

[0097] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects are apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but are intended to make the full scope consistent with the language claims, wherein the reference to the singular element is not intended to mean "only one", but to refer to "one or more", unless specifically noted. Unless otherwise specifically stated, the term "some" refers to one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are used only for convenience and do not limit the subject disclosure.

Claims

1. A method, comprising: accessing, by an electronic device, a plurality of lines of text data and text attributes corresponding to the plurality of lines of text data; for each corresponding line of the plurality of lines of the text data, determining, based on a first text attribute of the text attributes corresponding to the corresponding line of the plurality of lines and a second text attribute of the text attributes corresponding to a subsequent line of the plurality of lines, whether the corresponding line and the subsequent line correspond to separate paragraphs within the text data; generating output data for the plurality of lines, wherein the output data indicates which of the plurality of lines of the text data correspond to separate paragraphs; as well as At least one process is performed on the plurality of lines of the text data using the generated output data.

2. The method of claim 1, wherein accessing the text attributes comprises: An image including the plurality of lines of text data is received; and one or more bounding boxes are generated that are associated with one or more of the plurality of lines of text data. The method according to claim 1 , wherein the text attributes include semantic information and geometric information.

4. The method of claim 3, wherein the semantic information comprises one or more of punctuation, symbols, capitalization, word count, and part-of-speech tags.

5. The method of claim 3, wherein the geometric information comprises one or more of a line start position, a line height, a line spatial orientation, a line length, and a line spacing.

6. The method according to claim 1, further comprising: determining a language corresponding to the text data; as well as The determining is performed based on a reading order corresponding to the language. 7 . The method of claim 1 , wherein the output data includes data indicating that the corresponding line corresponds to a first paragraph and the subsequent line corresponds to a second paragraph.

8. The method of claim 1, wherein performing the at least one process on the plurality of lines of the text data using the generated output data comprises: modifying the plurality of lines of the text data using the output data; as well as The modified lines of the text data are copied to a clipboard.

9. The method of claim 1, wherein performing the at least one process on the plurality of lines of the text data using the generated output data comprises: The plurality of lines of the text data are copied to a clipboard in association with the output data.

10. The method of claim 1, wherein performing the at least one process on the plurality of lines of the text data using the generated output data comprises: The output data is provided to an application program or system process.

11. The method of claim 1 , wherein performing the at least one process on the plurality of lines of the text data using the generated output data comprises: The output data is provided to one or more of a text file, a data structure, a translation process, a dictation process, a narration process, and a virtual assistant.

12. The method of claim 1, wherein performing the at least one process on the plurality of lines of the text data using the generated output data comprises: The plurality of lines are translated according to the output data.

13. The method of claim 1, wherein performing the at least one process on the plurality of lines of the text data using the generated output data comprises: Audio data is generated for the plurality of lines based on the output data.

14. A method comprising: accessing, by an electronic device, a plurality of lines of text data and text attributes corresponding to the plurality of lines of text data; identifying a first list item row and a second list item row from the plurality of rows, wherein each of the first list item row and the second list item row begins with a list item indicator; generating list entries based on the first list item row and each corresponding row between the first list item row and the second list item row; for each respective one of the plurality of lines between the first list item line and the second list item line, determining, based on a first one of the text attributes corresponding to the respective one of the plurality of lines and a second one of the text attributes corresponding to a subsequent one of the plurality of lines, whether the respective one of the lines and the subsequent one of the lines correspond to separate paragraphs within the list item; generating output data for the plurality of lines, wherein the output data indicates which of the plurality of lines of the text data correspond to separate paragraphs; as well as At least one process is performed on the plurality of lines of the text data using the generated output data.

15. The method of claim 14, wherein accessing the text attributes comprises: An image including the plurality of lines of text data is received; and one or more bounding boxes are generated that are associated with one or more of the plurality of lines of text data. The method according to claim 14 , wherein the text attributes include semantic information and geometric information.

17. The method of claim 16, wherein the semantic information includes one or more of punctuation, symbols, capitalization, word count, and part-of-speech tags.

18. The method of claim 16, wherein the geometric information includes one or more of a line start position, a line height, a line spatial orientation, a line length, and a line spacing.

19. The method according to claim 14, further comprising: determining a language corresponding to the text data; as well as The determining is performed based on a reading order corresponding to the language.

20. A non-transitory computer readable medium comprising computer readable instructions that, when executed by a processor, cause the processor to perform operations comprising: accessing a plurality of lines of text data and text attributes corresponding to the plurality of lines of text data; for each corresponding line of the plurality of lines of the text data, determining, based on a first text attribute of the text attributes corresponding to the corresponding line of the plurality of lines and a second text attribute of the text attributes corresponding to a subsequent line of the plurality of lines, whether the corresponding line and the subsequent line correspond to separate paragraphs within the text data; generating output data for the plurality of lines, wherein the output data indicates which of the plurality of lines of the text data correspond to separate paragraphs; as well as At least one process is performed on the plurality of lines of the text data using the generated output data.

Images