SEGMENTATION OF TEXT TOPICS DERIVED FROM IMAGES

Abstract

This paper describes systems, methods, and other techniques for segmenting input text. A set of tokens is extracted from the input text. Token representations are computed for the token set. These token representations are then fed to a machine learning model, which generates a set of label predictions corresponding to the token set. The machine learning model was previously trained to generate label predictions in response to input token representations. Each of the label predictions indicates the position of a particular token within the token set relative to a particular segment. One or more segments within the input text are determined based on the set of label predictions.

Description

SEGMENTATION OF TEXT TOPICS DERIVED FROM IMAGES CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to United States Provisional Patent Application No. 63 / 009,185 filed on April 13, 2020, entitled “SEGMENTATION OF MESSY TEXTS- DETECTION OF THE BOUNDARIES OF TOPICALLY SIMILAR SEGMENTS IN TEXTS DERIVED FROM IMAGES OF HISTORICAL NEWSPAPERS”, the content of which is incorporated herein in its entirety. BACKGROUND OF THE INVENTION Text segmentation is the process of dividing a text into sections, with the granularity of the segmentation varying depending on the application. Types of text segmentation can include word segmentation, in which a text is divided into component words; sentence segmentation, in which a text is divided into component sentences; and topic segmentation, in which a text is divided into different topics. The task of topic segmentation may further include identifying (or categorizing) the specific topic for each of the divided segments. For example, both segmenting news feeds into articles on distinct topics and segmenting character sequences into words can be considered forms of text segmentation. There are several useful applications for text segmentation. For example, text segmentation can facilitate many downstream natural language processing tasks, including information extraction, text summarization, and passage retrieval. Topic segmentation, in particular, can be used to index documents, providing a specific portion of a document corresponding to a query as a result. Much of the previous work on topic segmentation has focused on segmenting clean blocks of narrative-style text, such as news articles or Wikipedia pages. Conventional approaches to these segmentation tasks detect boundaries between topics using unsupervised methods, for example, by measuring lexical cohesion or explicitly modeling topics, such as with Latent Dirichlet Assignment (LDA). More recently, supervised approaches have been shown to be more successful at detecting transitions between topics. Current state-of-the-art text segmentation methods use deep neural networks to predict whether a given sentence marks a segment boundary. Two types of approaches have dominated previous work on topic segmentation. The first approach is unsupervised and attempts to determine lexical, semantic, or topical similarity between adjacent sections of text. Contiguous sections that are very similar are considered to constitute a segment, and segment boundaries are detected by finding adjacent sections of text that are different. The second approach uses supervised machine learning methods that are trained on data labeled with segment boundaries. In some cases, these supervised models also take advantage of the fact that segments must be topical to solve the problem of identifying segment boundaries. Despite the progress made, conventional approaches to topic segmentation cannot produce the necessary levels of accuracy, making them unsuitable for many applications. Therefore, new systems, methods, and techniques for topic segmentation are needed. BRIEF DESCRIPTION OF THE INVENTION The methods described herein refer to techniques for segmenting text into different sections based on the topic of each section. Many methods frame the task as a token-level (block of text) sequence labeling problem, in which various representations are computed for each token in the text. While many methods are described with reference to a particular text segmentation task in which a newspaper marriage announcement is divided into units of one pair each, the methods are broadly applicable to any type of text that may contain different topics. A summary of the various embodiments of the invention is provided later as a list of examples. As used hereafter, any reference to a set of examples shall be understood as a reference to each of those examples disjunctively (e.g., Examples 1-4 shall be understood as Examples 1, 2, 3, or 4). Example 1 is a computer-implemented method for segmenting an input text, the method comprising: extracting a set of tokens from the input text; computing token representations for the set of tokens; providing the token representations to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided with input token representations, and wherein each of the set of label predictions indicates a position of a particular token from the set of tokens with respect to a particular segment; and determining one or more segments within the input text based on the set of label predictions. Example 2 is the computer-implemented method of Example 1, which further comprises: receiving an image; and generating the input text based on the image iviA / a / zuzz / ui ¿fox using a character recognition module. Example 3 is the computer-implemented method of Examples 1-2, wherein the computation of token representations for the set of tokens includes: calculating a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image. Example 4 is the computer-implemented method of Examples 1-3, wherein the character recognition module is an optical character reader. Example 5 is the computer-implemented method of Examples 1-4, wherein the position of the particular token with respect to the particular segment is one of: at the beginning of the particular segment; within the particular segment; or outside the particular segment. Example 6 is the computer-implemented method of Examples 1-5, where the image includes a plurality of marriage announcements captured from a newspaper. Example 7 is the computer-implemented method of Examples 1-6, where the machine learning model includes a bidirectional short-term long-term memory (LSTM) layer. Example 8 is the computer-implemented method of Examples 1-7, wherein the computation of token representations for the set of tokens includes at least one of: computing an ELMo embedding for each of the set of tokens using a trained ELMo model; or computing a GloVe embedding for each of the set of tokens using a trained GloVe mode. Example 9 is a computer-readable hardware storage device comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations to segment an input text, the operations comprising: extracting a set of tokens from the input text; computing token representations for the set of tokens; providing the token representations to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided with input token representations, and wherein each of the set of label predictions indicates a position of a particular token from the set of tokens with respect to a particular segment;and determine one or more segments within the input text based on the set of label predictions. Example 10 is the computer-readable hardware storage device from Example 9, where the operations also include: receiving an image; and generating the input text based on the image using a character recognition module. Example 11 is the computer-readable hardware storage device MA / a / ZUZZ / Ul Z lo» of Examples 9-10, wherein calculating token representations for the set of tokens includes: calculating a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image. Example 12 is the computer-readable hardware storage device of Examples 9-11, wherein the character recognition module is an optical character reader. Example 13 is the computer-readable hardware storage device of Example 9-12, wherein the position of the particular token with respect to the particular segment is one of: at the beginning of the particular segment; within the particular segment; or outside the particular segment. Example 14 is the computer-readable hardware storage device of Example 9-13, where the image includes a plurality of marriage announcements captured from a newspaper. Example 15 is the computer-readable hardware storage device of Examples 9-14, where the machine learning model includes a bidirectional short-term long-term memory (LSTM) layer. Example 16 is the computer-readable hardware storage device of Examples 9-15, wherein the computation of token representations for the set of tokens includes at least one of: computing an ELMo embedding for each of the set of tokens using a trained ELMo model; or computing a GloVe embedding for each of the set of tokens using a trained GloVe mode. Example 17 is a system for segmenting an input text, the system comprising: one or more processors; and a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: extracting a set of tokens from the input text; computing token representations for the set of tokens; providing the token representations to a machine-learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine-learning model was previously trained to generate label predictions in response to being provided with input token representations, and wherein each of the set of label predictions indicates a position of a particular token from the set of tokens with respect to a particular segment;and determine one or more segments within the input text based on the set of label predictions. Example 18 is the system of Example 17, where the operations also include: receiving an image; and generating the input text based on the image using a IVIA / a / ZUZZ / UI 4 / 0» character recognition module. Example 19 is the system of Examples 17-18, wherein the calculation of token representations for the set of tokens includes: calculating a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image. Example 20 is the system of Example 17-19, where the position of the particular token with respect to the particular segment is one of: at the beginning of the particular segment; within the particular segment; or outside the particular segment. BRIEF DESCRIPTION OF THE FIGURES The accompanying drawings, included to provide a greater understanding of disclosure, are incorporated into and form part of this specification. They illustrate modalities of disclosure and, together with the detailed description, serve to explain the principles of disclosure. No attempt is made to show structural details of disclosure in more detail than may be necessary for a fundamental understanding of disclosure and the various ways in which it can be implemented. FIGURE 1 illustrates an example of a text segmentation system. FIGURE 2 illustrates an example architecture of a text segmentation system. FIGURE 3 illustrates an example architecture of portions of a segmentation pipeline. FIGURE 4 illustrates an example training scheme for training a machine learning model of a segmentation pipeline. Figures 5A and 5B illustrate an example of precise text segmentation that can be produced by a segmentation pipeline. Figures 6A and 6B illustrate an example of precise text segmentation that can be produced by a segmentation pipeline. Figures 7A and 7B illustrate example results from an evaluation study of a segmentation pipeline and the corresponding model. FIGURE 8 illustrates a method for training a machine learning model of a segmentation pipeline. FIGURE 9 illustrates a method for segmenting an input text. FIGURE 10 illustrates an example computer system comprising several hardware elements. DETAILED DESCRIPTION OF THE INVENTION The methods described herein refer to a level of text segmentation known as topic segmentation, which is the task of dividing a text into sections with distinct thematic content. As used herein, the segmentation of MA / a / ZUZZ / Ul Z lo» themes can refer both to the case of dividing a text into different instances of the same general theme (for example, dividing a text into a first marriage announcement, a second marriage announcement and a third marriage announcement) and to the case of dividing a text into completely different themes (for example, dividing a text into a marriage announcement, an obituary and an advertisement). The modalities described herein include a novel deep learning-based model and a segmentation pipeline for segmenting jumbled text derived from images that significantly outperforms existing state-of-the-art techniques. The segmentation pipeline can be used on text that lacks narrative structure and exhibits topical similarity between segments. In some modalities, boundaries between segments are predicted at the token level rather than at the sentence or paragraph level. In some modalities, the image-to-text conversion software can provide the physical location of each token, which can be used as a feature of the pipeline. This aids in detecting segment beginnings, as they often start at the beginning of a new line of text. In some modalities, ELMo embeddings are used as a pipeline feature. The language model from which ELMo embeddings are generated can be fine-tuned on a large corpus of newspaper text obtained from optical character recognition (OCR). This fine-tuning allows the ELMo model to generate newspaper-specific embeddings and embeddings that capture the meanings of words with common OCR errors. In some modalities, given the information hierarchy within an input text, the task is not approached as a strictly linear segmentation. Instead of simply predicting the boundaries between segments, it is possible to predict whether each token is at the beginning of a segment, within a segment, or outside of a segment. In the context of wedding announcements, the described segmentation system can be used in conjunction with a sequence tagging model trained to tag key wedding facts, such as: (1) the bride (Bride), (2) the groom (Groom), (3) the wedding date (WeddingDate), (4) the wedding venue (WeddingVenue), (5) the bride's residence (BrideResidence), (6) the groom's residence (GroomResidence), among other possibilities. Information that falls outside a segment, such as information tagged WeddingDate, can be assumed to apply to all couples after that segment, until another WeddingDate tag is reached or the article ends. For the sake of brevity, many of the methods described herein are illustrated with reference to a specific example: the segmentation of marriage announcements found in historical newspapers. In some cases, the text derived from images that The text "MA / a / ZUZZ / Ul Z lo," which includes marriage announcements, can exhibit many properties that make it unsuitable for segmentation using existing techniques. For example, the text may not be structured into sentences, and adjacent segments may not be topically distinct from one another. Furthermore, the text in the announcements, which is obtained from historical newspaper images via OCR, may contain numerous typographical errors. However, the methods described in this disclosure are applicable to a wide range of applications and are not limited to the specific examples described. For example, methods described in this disclosure may utilize historical or periodical documents containing information such as obituaries, divorce lists, birth announcements, real estate transactions, advertisements, sports scores, receipts, song lyrics, moving captions, recipes, and other possibilities. The following description will provide several examples. For explanatory purposes, specific configurations and details are provided to ensure a complete understanding of the examples. However, it will also be evident to someone proficient in the technique that the example can be practiced without these specific details. Furthermore, well-known features may be omitted or simplified to avoid complicating the described methods. Figure 1 illustrates an example of a system 100 for segmenting text, according to some embodiments of the present disclosure. The system 100 may include a segmentation pipeline 110 that receives input text 112 as input and generates segments 114 as output based on the input text 112. The segments 114 may correspond to the input text 112 and may indicate which portions of the input text 112 belong to each of the segments 114. For example, a first portion of input text 112 may be segmented into a first segment of segments 114, a second portion of input text 112 may be segmented into a second segment of segments 114, and a third portion of input text 112 may not belong to any of the segments 114. In some configurations, System 100 may also include a text converter 118 that converts a text source 116 into input text 112. In some examples, the text source 116 may be an image containing text, and the text converter 118 may be an optical character reader that performs OCR on the image to extract / read the text. In some examples, the text source 116 may be an audio or voice signal, and the text converter 118 may be an audio-to-text converter. In several examples, the text source 116 may be an image, a video, an audio signal, previously extracted text, a handwriting signal (detected by an electronic writing device), among other possibilities. Figure 2 illustrates an example architecture of a 200 system for segmentation MA / a / dUdd / Ul ¿IOX text, according to some modalities of this disclosure. Similar to system 100, system 200 may include a segmentation pipeline 210 that receives an input text 212 and outputs a set of segments 214 based on the input text 212. The segmentation pipeline 210 may include one or more machine learning models, such as model 220, which can be trained using a training dataset before deploying system 200 in a runtime scenario. Optionally, system 200 may include a text converter, such as a character recognition module 218, which converts a text source, such as an image 216, into input text 212. During system runtime or training 200, the input text 212 can be parsed to extract a set of tokens. For each token ti in the token set, various token representations can be computed using several models and vector generators. The representations for all the tokens can form token representations 222. For example, the input text tokens 212 can be fed to an ELMo / BERT model 228, which can generate ELMo embeddings using a pre-trained language model. In some examples, each token can be fed to a pre-trained GloVe model 230, which can produce non-contextual embeddings for each input text token 212. In some examples, each token can be fed to a learned character model 232, which can produce learned character-level embeddings for each input text token 212. In some examples, the input text tokens 212 can be provided to a typography vector generator 234, which can generate a typography vector that is a one-hot encoded representation conveying the geometry of the tokens in the input text 212 (e.g., uppercase, lowercase, mixed uppercase, alphanumeric, special characters, etc.). In some examples, each token can be provided to a position vector generator 236, which can generate a position vector for each input text token 212 indicating the position of each token relative to a physical reference point within the image 216. As an example, in some implementations, for each learned token, the 232 character model can first compute a character-based representation. This can be achieved by representing each character in the token as a 25-dimensional learned embedding. The character embeddings can then be passed through a convolutional layer consisting of 30 three-dimensional (3D) filters, using the same padding (e.g., a preprocessing step where dummy characters are added to ensure that multiple sequences have the same length when fed into the model), and followed by global maximum pooling. The output of the maximum pooling can then be concatenated with externally trained token embeddings. In some implementations, for each token q, a 100-dimensional GloVe embedding can be employed. In some implementations, for each token q, an ELMo embedding generated from the pre-trained language model can be used. This embedding can be tuned for one or more epochs on a corpus of billions of tokens obtained from newspaper page images using OCR. In some implementations, the ELMo model can return a set of three embeddings for each token. A weighted average of these three embeddings can be computed, with weighting factors learned during training, to obtain tfmo. Since all tokens may be lowercase before use, an eight-dimensional one-hot encoded vector t151 can be embedded to represent the original capitalization of each token (e.g., uppercase, lowercase, mixed, etc.). In some implementations, the position vector generator 236 can generate position vectors that are indicative of the physical locations of tokens. For example, the character recognition module 218 can employ OCR software that produces a bounding box for each token, including the x and y pixel coordinates of the upper-left corner of the token's bounding box. These physical locations can be a potential signal for the start of a new line of printed text. Since the raw x and y coordinates can vary drastically depending on which part of an image an item originates from, a distance vector between tokens can be calculated using the x and y coordinates of each token. The distance vector can be set to [0, 0] for the first token in an item. The complete embedding (token representations) for the thvL token is given by Equation (1) for implementations that do not make use of distance vectors and by Equation (2) for implementations that can use distance vectors. In both equations, o denotes concatenation. P _ ^.charo^glove θ ^elme0^.casing P _ ^char q ^glove θ ^.elmoo^.casing θ ^dist When provided with token representations 222 (v¿), model 220 can generate a set of label predictions 226 that includes a label prediction for each token in the set. As shown in the example placed above the system architecture in FIGURE 2, a label prediction of “Marriage B” can be generated for the token “Howard,” and a label prediction of “Marriage I” can be generated for each of the tokens “E.”, “Rustles,” and “38.” The label predictions ¿fox Label predictions 226 can be used by a segment extractor 224 to identify segments 214. For example, each label prediction 226 can indicate (1) whether a particular token belongs to one of the segments 214 and, if so, (2) the position of the particular token within the segment to which it belongs. For example, the label prediction Marriage B indicates that the token Howard belongs to a particular segment with the theme of Marriage and furthermore that the token is at the beginning (indicated by B) of the particular segment. As another example, the label prediction Marriage I indicates that the token E belongs to a particular segment with the theme of Marriage and furthermore that the token is within (indicated by I) of the particular segment (i.e., not at the beginning of the segment). A label prediction of O would indicate that a particular segment is outside of all segments. Figure 3 illustrates an example architecture of portions of a segmentation pipeline 310, according to some modalities of this disclosure. In the example in Figure 3, input text 312 containing tokens of MARRIED, Amy, Lee, and 25 is fed to segmentation pipeline 310, which computes token representations 322 based on the tokens. The token representations 322 are then fed into model 320, which may include a bidirectional short-term-long-term memory (LSTM) layer and a conditional random field (CRF) layer arranged sequentially as shown. Model 320 can then generate 326 tag predictions for the tokens as shown, which include a tag prediction of “O” for the “MARRIED” token, a tag prediction of “Marriage B” for the “Amy” token, and a tag prediction of “Marriage I” for each of the “Lee” and “25” tokens. In some implementations, the sequence of all token representations (token embeds) for a document can have a length n, and the token representations vi:n can be passed through a single BiLSTM layer, with a particular state size (e.g., 100) for each direction. Dropout can be applied at a certain rate (e.g., 0.5) before feeding the token representations to the BiLSTM layer of the 320 model. Let LSTM and LSTM denote the forward and backward LSTMs, respectively, and let q and q denote the internal cell states of the forward and backward LSTMs at position i, respectively. A hidden representation h1 for each token can be obtained as follows: Kc¡= LSTM (3) hl,c~l= LSTM(vi,hl_1,c[~[) (4) hí =K °K (5) In some implementations, the hidden output sequence of the BiLSTM layer h1:rawiAiaidVddivi ​​¿iox can then be fed as input to a linear string CRF to produce an output sequence of labels y1:n. During inference, the Viterbi algorithm can be used to decode the most probable sequence y1;n. Figure 4 illustrates an example training scheme for training a machine learning model 420 from a segmentation pipeline 410, according to some modalities of this disclosure. During each training iteration, an image 416 is taken from the training dataset 444 (which may include a large number of images to be used as training examples) and provided to a hand labeler 446 and a character recognition module 418. The hand labeler 446 can analyze the image 416 and identify a set of fundamental truth segments 442. In response to receiving the image 416, the character recognition module 418 can generate input text 412 based on the image 416. Furthermore, during each training iteration, input text 412 can be provided to the segmentation pipeline 410 to generate segments 414 using model 420. For example, as described with reference to Figures 2 and 3, model 420 can be used to generate label predictions that can be used to determine segments 414 within the input text 412. Segments 414 can be compared to the fundamental truth segments 442 using the loss calculator 438 to calculate a loss 440, which can be used to adjust the weighting factors associated with model 420. In some modalities, the loss 440 can be used to adjust the weighting factors associated with model 420 so that the loss 440 is reduced during subsequent iterations using Figure 416. The training steps described above can be repeated for each image 416 in training dataset 444. In some modalities, the entire training dataset 444 can be used for training, while in others, a portion of training dataset 444 can be used for training and the remaining portion for evaluation. In some modalities, multiple training epochs can be performed so that specific training examples (e.g., images) from training dataset 444 can be used multiple times. Other possibilities are being considered. In some implementations, the model weighting factors of model 420 are trained in a supervised manner following the negative log probability loss function L for loss 440, which can be defined as follows: = -Zfcl0gp(yi:nJh1:n / c) (6) ινΐΛ / a / zuzz / uiz / oa In Equation (6), p(yi:T¡Jhi:nk) denotes the probability assigned by the CRF to the true label sequence y1;n for the training example k with nktokens given a hidden output sequence of the BiLSTM layer h1;n. In some implementations, a mini-batch gradient descent with some batch size (e.g., 16) can be employed using, for example, nadam's optimizer (with, for example, a learning rate of 0.001). Figures 5A and 5B illustrate an example of accurate text segmentation that can be produced by a segmentation pipeline, in accordance with some modalities of this disclosure. Figure 5A shows an image 516 containing text that can be provided to the segmentation pipeline, and Figure 5B shows segments 514 that can be generated by the segmentation pipeline. Despite the complexity of the text (e.g., certain words are broken across multiple lines, errors in the text, etc.), the segments 514 are an accurate topic segmentation for image 516. Figures 6A and 6B illustrate an example of accurate text segmentation that can be produced by a segmentation pipeline, according to some modalities of this disclosure. Figure 6A shows an image 616 containing text that can be provided to the segmentation pipeline, and Figure 6B shows segments 614 that can be generated by the segmentation pipeline. Although the text is jumbled with multiple errors due to the quality and low resolution of image 616, the segments 614 are an accurate topic segmentation for image 616. Figures 7A and 7B illustrate example results from an evaluation study of the segmentation pipeline and the corresponding model, according to some modalities of this disclosure. In the evaluation study, the objective was to segment the text of newspaper marriage announcements into one-couple segments. The dataset used included newspapers from English-language publications spanning the years 1824–2018. The entire dataset consisted of 1,384 newspaper articles, of which 1,179 were extracted from newspaper page images by OCR using commercial software. The articles contained a total of 16,833 segments, and the average number of segments per article was 7. The average document length was 610 characters, and the average segment length was 63 characters. The core truth segments were manually labeled.The dataset was divided into training, development, and testing datasets of 804, 303, and 277 items, respectively. Each token was tagged with one of the tags “Marriage B”, “Marriage I”, or “O”, indicating, respectively, that the token marks the beginning of a wiAiaidVddivi ​​fox marriage ad segment, is within a segment, or is not in a segment. This tagging scheme allows the targeting pipeline to simultaneously segment text and categorize segments. For comparison with previous work, the Pkmetric was calculated, which is the probability that, when sliding a window of size k over predicted segments, the window's endpoints will be in different segments when they should have been in the same segment, or vice versa. For the Pk calculation, all tokens must be included in a segment. Before the Pk calculation, any O tags in the predictions and the fundamental truth were converted to Marriage B or “Marriage B” (as appropriate, depending on their position), so that stretches of O tags became segments. For the Pk calculation, k was set to half the average segment size for each document. Because standard segmentation evaluation metrics do not account for the fact that some errors are worse than others, an alternative way to measure segmentation accuracy was developed as a task-based evaluation method. For all marriage listings in the test dataset, a set of marriage-related entities (Bride, Groom, Wedding Venue, Wedding Date, etc.) was manually labeled. These entities were used in the task-based evaluation as follows. Core truth segments were iterated to find the predicted segment with the greatest overlap, and a continuous count was maintained of: (1) all entities in the core truth segment (expected), (2) expected entities found in the best-matched predicted segment (found), and (3) any entities included in the predicted segment but not in the core truth segment (extra).Accuracy and recovery were then calculated as follows: Precision(found, extra) = —found— found+ Recovery(found, expected) = (8) Table 700 in Figure 7A shows the results of the segmentation pipeline experiments described with various features, compared to the recent model proposed by Koshorek et al. The segmentation pipeline significantly outperforms the Koshorek model, as measured by both Punkt and the task-based evaluation method. The Koshorek model shows particularly low accuracy in the task-based evaluation, indicating a tendency to undersegment. This is consistent with the observation that the Punkt sentence tokenizer used by the Koshorek model tends to group multiple marriage advertisements together as a segment. Across the entire dataset, 48% of all segment boundaries were found not to align with the sentence boundaries identified by the Punkt sentence tokenizer. Table 700 also shows experiments to determine the contribution of ELMo embeddings, token positions, GloVe embeddings, and BIO encoding to segmentation pipeline performance. The best performance, as measured by the highest F1 score in the task-based evaluation, was achieved when ELMo embeddings, GloVe embeddings, and token position vectors were included as features, and when Marriage B, Marriage I, and O were used as token labels. The use of ELMo embeddings increased the F1 score by more than 4% (from 93.4 to 97.7). A significant portion of this increase can be attributed to the fine-tuning of the ELMo language model in the same-domain text. Without fine-tuning, the F1 score was 95.5, while with fine-tuning, the F1 score was 97.7. The contribution of token position vectors was smaller, increasing the F1 score from 97.1 without position vectors to 97.7 with position vectors. As noted earlier, three token labels (Marriage B, Marriage I, and O) were used because certain sections of the text may not be part of any marriage announcement. This differs from approaches where all parts of the document are assumed to belong to a segment, and the task is formulated as finding either the beginning or the end of each segment. This can be referred to as a B1 labeling scheme, whereas the use of three token labels can be referred to as a B1 labeling scheme. Koshorek's model uses an approach where each sentence is labeled as either the end of a segment or not the end of a segment. This is not technically a B1 scheme since it predicts segment ends rather than beginnings; however, only two class labels are predicted.For a more direct comparison of the segmentation pipeline, a Bl-tagged version of the data was created, in which any token tagged with “O” became either “Marriage B” or “Marriage I”. The training and testing results on this converted dataset are shown in Tables 700 and 702 in Figures 7A and 7B, respectively. Using three token labels instead of two appeared to improve performance as measured by Pk and improve recall in task-based evaluation, while decreasing accuracy and F1 in task-based evaluation. Perhaps unsurprisingly, performance as measured by Pk is improved when using the Bl labeling scheme. As noted earlier, to calculate Pk for predictions labeled with BIO, they were converted to a Bl labeling scheme, since the Pk calculation assumes that each token is part of a segment. This conversion was not necessary when the model was trained on Bl-labeled tags.As such, when training on Bl-labeled data, the training objective aligned more closely with the Pk evaluation metric than when training on BIO-labeled data. It can be argued that task-based evaluation is a more meaningful metric for this use case, as it quantifies errors that would directly impact information extraction. Using this metric, it can be observed that the BIO labeling scheme achieves greater accuracy and a higher F1 score than the Bl labeling scheme. Table 702 shows a more detailed analysis of the task-based evaluation. Performance for WeddingDate, a fact typically located outside of marriage announcement segments, shows a significant increase when the O tags are used in addition to the Marriage B and Marriage I tags. It can be speculated that the inclusion of the O tag allows the model to specialize further, perhaps by learning specific features associated with non-wedding text, and thus allows the model to do a better job of excluding those sections from adjacent segments. Figure 8 illustrates an 800 method for training a machine learning model (e.g., models 220, 320, 420) from a segmentation pipeline (e.g., segmentation pipelines 110, 210, 310, 410), according to some embodiments of this disclosure. One or more steps of the 800 method may be omitted during the execution of the 800 method, and the steps of the 800 method may be performed in any order and / or in parallel. One or more steps of the 800 method may be performed by one or more processors or may be implemented as a computer-readable medium or computer program product comprising instructions that, when the program is executed by one or more computers, cause the one or more computers to carry out the steps of the 800 method. In step 802, an image (for example, images 216, 416, 516, 616) is retrieved from a training dataset (for example, training dataset 444) and provided to a character recognition module (for example, character recognition modules 218, 418) and a manual labeler (for example, manual labeler 446). The character recognition module can generate input text (for example, input texts 112, 212, 312, 412) based on the image. The manual labeler can produce one or more fundamental truth segments (for example, fundamental truth segment 442) by analyzing the image. In step 804, the input text is provided to the segmentation pipeline to generate one or more segments (for example, segments 114, 214, 414, 514, 614) within the input text. The segmentation pipeline, which includes the machine learning model, can be used to generate the one or more segments based on the input text. The machine learning model can be associated with multiple weighting factors. In step 806, a loss (e.g., loss 440) is calculated based on a comparison between the one or more fundamental truth segments and the one or more segments generated by the segmentation pipeline. In step 808, the plurality of weighting factors associated with the machine learning model is adjusted using loss. Figure 9 illustrates a Method 900 for segmenting an input text (for example, input texts 112, 212, 312, 412), according to some embodiments of this disclosure. One or more steps of Method 900 may be omitted during the execution of Method 900, and the steps of Method 900 may be performed in any order and / or in parallel. One or more steps of Method 900 may be performed by one or more processors or may be implemented as a computer-readable medium or computer program product comprising instructions that, when the program is executed by one or more computers, cause the one or more computers to perform the steps of Method 900. In some embodiments, Method 900 may be performed after performing Method 800. In step 902, a set of tokens is extracted from the input text. The input text can be generated from an image (e.g., images 216, 416, 516, 616) using a character recognition module (e.g., character recognition modules 218, 418). The character recognition module can be an optical character reader. The image can be retrieved from a dataset (e.g., training dataset 444). The image can contain a plurality of marriage announcements captured from a newspaper. In step 904, token representations (for example, token representations 222, 322) are calculated for the token set. Calculating the token representations for the token set can include calculating an ELMo embedding for each token in the set using a trained ELMo model, calculating a GloVe embedding for each token in the set using a trained GloVe mode, and / or calculating a position vector for each token in the set, among other possibilities. The position vector can indicate a token's location relative to a physical reference point within the image. In step 906, token representations are provided to a machine learning model (for example, models 220, 320, 420) that generates a set of label predictions (for example, label predictions 226, 326) corresponding to the set of tokens. The machine learning model may have been previously trained to generate label predictions in response to token representations being provided as input. Each of the set of label predictions can indicate a position of a particular token within the set of tokens relative to a particular segment. The position of the particular token relative to the particular segment can IVIA / a / ZUZZ / UI can be at the beginning of the particular segment, within the particular segment, or outside the particular segment. The machine learning model can include a bidirectional LSTM layer. The machine learning model can be an element of a segmentation pipeline (for example, segmentation pipelines 110, 210, 310, 410). In step 908, one or more segments (for example, segments 114, 214, 414, 514, 614) within the input text are determined based on the tag prediction set. These segments can be determined based on the token set positions indicated by the tag prediction set. Figure 10 illustrates an example computer system 1000 comprising several hardware elements, according to some modalities of this disclosure. The computer system 1000 can be incorporated or integrated into devices described herein and / or can be configured to perform some or all of the steps of the methods provided by various modalities. For example, in various modalities, the computer system 1000 can be configured to perform methods 800 and 900. It should be noted that Figure 10 is intended only to provide a generalized illustration of various components, any or all of which can be used as appropriate. Figure 10, therefore, broadly illustrates how the individual elements of the system can be implemented in a relatively separate or relatively more integrated manner. In the illustrated example, computer system 1000 includes a communication medium 1002, one or more processors 1004, one or more input devices 1006, one or more output devices 1008, a communication subsystem 1010, and one or more memory devices 1012. Computer system 1000 can be implemented using various hardware implementations and embedded system technologies. For example, one or more elements of computer system 1000 can be implemented as a field-programmable gate array (FPGA), such as those commercially available from XILINX®, INTEL®, or LATTICE SEMICONDUCTOR®, a system-on-a-chip (SoC), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a microcontroller, and / or a hybrid device, such as an FPGA SoC, among other possibilities. The various hardware elements of computer system 1000 can be coupled by means of communication medium 1002. While communication medium 1002 is illustrated as a single connection for clarity, it should be understood that communication medium 1002 can include various numbers and types of communication media for transferring data between hardware elements. For example, communication medium 1002 can include one or more wires (e.g., conductive traces, paths, or conductors on a printed circuit board (PCB) or integrated circuit (IC), microstrips, strip lines, cables). WUUa / dUdd / Ul dlo» coaxial), one or more optical waveguides (e.g., optical fibers, strip waveguides) and / or one or more wireless connections or links (e.g., infrared wireless communication, radio communication, microwave wireless communication), among other possibilities. In some configurations, the communication medium 1002 may include one or more buses connecting pins of the computer system hardware elements 1000. For example, the communication medium 1002 may include bus-connected processors 1004 to main memory 1014, called the system bus, and a bus connecting main memory 1014 to input devices 1006 or output devices 1008, called the expansion bus. The system bus may consist of several elements, including an address bus, a data bus, and a control bus. The address bus may carry a memory address from the processors 1004 to the address bus circuitry associated with main memory 1014 so that the data bus can access and carry the data contained at that memory address back to the processors 1004. The control bus may carry commands from the processors 1004 and return status signals from main memory 1014.Each bus can include multiple wires to carry multiple bits of information, and each bus can support serial or parallel data transmission. The 1004 processor(s) may include one or more central processing units (CPUs), graphics processing units (GPUs), neural network processors or accelerators, digital signal processors (DSPs), and / or similar components. A CPU may take the form of a microprocessor, which is manufactured on a single-chip IC constructed from metal-oxide-semiconductor field-effect transistors (MOSFETs). The 1004 processor(s) may include one or more multi-core processors, in which each core can read and execute program instructions simultaneously with the other cores. Input devices 1006 may include one or more user input devices such as a mouse, keyboard, microphone, and various sensor input devices, such as an image capture device, a pressure sensor (e.g., a barometer, a touch sensor), a temperature sensor (e.g., a thermometer, a thermocouple, a thermistor), a motion sensor (e.g., an accelerometer, a gyroscope, a tilt sensor), a light sensor (e.g., a photodiode, a photodetector, a charge-coupled device), and / or similar devices. Input devices 1006 may also include devices for reading and / or receiving removable storage devices or other removable media.These removable media may include optical discs (e.g., Blu-ray discs, DVDs, CDs), memory cards (e.g., CompactFlash card, Secure Digital (SD) card, Memory Card), floppy disks, Universal Serial Bus (USB) flash drives, external hard disk drives (HDDs) or solid-state drives (SSDs) and / or similar. Output devices 1008 may include one or more devices that convert information into a human-readable form, such as, but not limited to, a display device, a loudspeaker, a printer, and / or similar devices. Output devices 1008 may also include devices for writing to removable storage devices or other removable media, such as those described in reference to input devices 1006. Output devices 1008 may also include various actuators for causing physical movement of one or more components. These actuators may be hydraulic, pneumatic, or electric and may be provided with control signals by the computer system 1000. The communications subsystem 1010 may include hardware components for connecting the computer system 1000 to systems or devices located on the external computer system 1000, such as through a computer network. In various configurations, the communications subsystem 1010 may include a wired communication device coupled to one or more input / output ports (for example, a universal asynchronous receiver-transmitter (UART)), an optical communication device (for example, an optical modem), an infrared communication device, a radio communication device (for example, a wireless network interface controller, a BLUETOOTH® device, an IEEE 802.11 device, a Wi-Fi device, a WiMAX device, a cellular device), among other possibilities. Memory devices 1012 can include the various data storage devices of the computer system 1000. For example, memory devices 1012 can include various types of computer memory with varying response times and capacities, ranging from faster response times and lower capacity memory, such as processor registers and cache memories (e.g., LO, L1, L2), to medium response time and medium capacity memory, such as random access memory, to slower response times and lower capacity memory, such as solid-state drives and hard disks. While processors 1004 and memory devices 1012 are illustrated as separate items, it should be understood that processors 1004 can include varying levels of on-processor memory, such as processor registers and cache memories that can be used by a single processor or shared among multiple processors. Memory devices 1012 may include main memory 1014, which can be directly accessed by processors 1004 via the memory bus of the communication medium 1002. For example, processors 1004 can continuously read and execute instructions stored in main memory 1014. As such, various elements of MA / a / ZUZZ / Ul Z lo» software can be loaded into main memory 1014 to be read and executed by the processors 1004 as illustrated in FIGURE 10. Typically, main memory 1014 is volatile memory, which loses all data when power is turned off and, consequently, requires power to preserve the stored data. Main memory 1014 may also include a small portion of software containing non-volatile memory (for example, firmware, such as BIOS) that is used to read other software stored on memory devices 1012 into main memory 1014.In some forms, the volatile memory of the main memory 1014 is implemented as random access memory (RAM), such as dynamic RAM (DRAM), and the non-volatile memory of the main memory 1014 is implemented as read-only memory (ROM), such as flash memory, erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM). Computer system 1000 may include software elements, currently shown as residing in main memory 1014, which may include an operating system, device driver(s), firmware, compiler, and / or other code, such as one or more application programs, which may include computer programs provided by various means of this disclosure. By way of example only, one or more steps described with respect to any method discussed above could be implemented as instructions 1016, executable by computer system 1000.In one example, these instructions 1016 can be received by the computer system 1000 using the communications subsystem 1010 (for example, via a wireless or wired signal carrying instructions 1016), carried by the communication medium 1002 to the memory device(s) 1012, stored within the memory device(s) 1012, read into main memory 1014, and executed by the processors 1004 to perform one or more steps of the methods described. In another example, instructions 1016 can be received by the computer system 1000 using input devices 1006 (for example, through a removable media reader), ported over communication medium 1002 to memory devices 1012, stored within memory devices 1012, read into main memory 1014, and executed by processors 1004 to perform one or more steps of the methods described. In some embodiments of this disclosure, instructions 1016 are stored on a computer-readable storage medium or simply a computer-readable medium. This computer-readable medium may be non-transient and may therefore be referred to as a non-transient computer-readable medium. In some cases, the non-transient computer-readable medium may be incorporated within the computer system 1000. For example, the non-transient computer-readable medium may be one of the memory devices 1012, as shown in FIGURE 10, with instructions 1016 being stored within the memory devices 1012. In some cases, the non-transient computer-readable medium may be separate from the computer system 1000.In one example, the non-transient computer-readable medium may be a removable medium provided to input devices 1006 such as those described with reference to input devices 1006, as shown in FIGURE 10, with instructions 1016 provided to input devices 1006. In another example, the non-transient computer-readable medium may be a component of a remote electronic device, such as a mobile phone, which can wirelessly transmit a data signal carrying instructions 1016 to computer system 1000 using communications subsystem 1016, as shown in FIGURE 10, with instructions 1016 provided to communications subsystem 1010. Instructions 1016 can take any form suitable for being read and / or executed by computer system 1000. For example, instructions 1016 can be source code (written in a human-readable programming language such as Java, C, C++, C#, Python), object code, assembly language, machine code, microcode, executable code, and / or similar formats. In one example, instructions 1016 are provided to computer system 1000 as source code, and a compiler is used to translate the instructions 1016 from source code into machine code, which can then be read into main memory 1014 for execution by processors 1004. As another example, instructions 1016 are provided to computer system 1000 as an executable file containing machine code that can be immediately read into main memory 1014 for execution by processors 1004.In several examples, the 1016 instructions can be provided to the computer system 1000 in encrypted or unencrypted form, in compressed or uncompressed form, as an installation package or an initialization for a wider software implementation, among other possibilities. In one aspect of this disclosure, a system (for example, computer system 1000) is provided for carrying out methods according to various embodiments of this disclosure. For example, some embodiments may include a system comprising one or more processors (for example, processors 1004) communicatively coupled to a non-transient computer-readable medium (for example, memory devices 1012 or main memory 1014). The non-transient computer-readable medium may have instructions (for example, instructions 1016) stored therein that, when executed by the one or more processors, cause the one or more processors to carry out the methods described in the various embodiments. In another aspect of this disclosure, a program product is provided MA / a / ZUZZ / Ul Z lo» computer that includes instructions (for example, instructions 1016) to perform methods according to various modalities of this disclosure. The computer program product can be tangibly embodied in a non-transient, computer-readable medium (for example, memory devices 1012 or main memory 1014). The instructions can be configured to cause one or more processors (for example, processors 1004) to perform the methods described in the various modalities. In another aspect of this disclosure, a non-transient computer-readable medium (for example, memory devices 1012 or main memory 1014) is provided. The non-transient computer-readable medium may have instructions (for example, instructions 1016) stored therein which, when executed by one or more processors (for example, processors 1004), cause the one or more processors to perform the methods described in the various modes. The methods, systems, and devices discussed above are examples. Different configurations may omit, substitute, or add various procedures or components as appropriate. For example, in alternative configurations, the methods may be performed in a different order than described, and / or various steps may be added, omitted, and / or combined. Furthermore, the features described with respect to certain configurations may be combined in several other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Moreover, technology evolves, and therefore many of the elements are examples and do not limit the scope of the disclosure or the claims. Specific details are provided in the description to provide a complete understanding of the example configurations, including their implementations. However, the configurations can be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail to avoid complicating the configurations. This description provides only example configurations and does not limit the scope, applicability, or configurations of the claims. Instead, the above description of the configurations will provide those skilled in the art with an enabling description for implementing the described techniques. Various changes to the function and arrangement of elements can be made without departing from the spirit or scope of the disclosure. Having described several examples of configurations, various modifications, alternative constructions, and equivalents can be used without deviating from the spirit of the disclosure. For example, the above elements may be components of a larger system, where other rules may take precedence over or otherwise modify the application of the technology. Furthermore, various steps may be carried out before, during, or after considering the above elements. Consequently, the above description does not MA / a / 4U44 / U1 4 / 0» limits the scope of the claims. As used herein and in the appended claims, the singular forms a, an, and the include plural references unless the context clearly indicates otherwise. Thus, for example, a reference to “a user” includes a reference to one or more such users, and a reference to “a processor” includes a reference to one or more processors and their equivalents known to those skilled in the art, and so forth. Furthermore, the words “comprising”, “comprising”, “containing”, “including”, “including” and “includes”, when used in this specification and in the following claims, are intended to specify the presence of indicated features, whole numbers, components or steps, but do not exclude the presence or addition of one or more features, whole numbers, components, steps, acts or groups. It is also understood that the examples and modalities described herein are for illustrative purposes only and that various modifications or changes will be suggested to those skilled in the art taking them into account and will be included within the spirit and scope of this application and the scope of the attached claims.

Claims

1. A computer-implemented method for segmenting an input text, the method being characterized in that it comprises: extracting a set of tokens from the input text; computing token representations for the set of tokens; providing the token representations to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided with input token representations, and wherein each of the set of label predictions indicates a position of a particular token from the set of tokens with respect to a particular segment; and determining one or more segments within the input text based on the set of label predictions.

2. The computer-implemented method according to claim 1, characterized in that it further comprises: receiving an image; and generating the input text based on the image using a character recognition module.

3. The computer-implemented method according to claim 2, characterized in that the calculation of the token representations for the set of tokens includes: calculating a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image.

4. The computer-implemented method according to claim 2, characterized in that the character recognition module is an optical character reader.

5. The computer-implemented method according to claim 4, characterized in that the position of the particular token with respect to the particular segment is one of: at the beginning of the particular segment; within the particular segment; or outside the particular segment. MA / a / dUdd / Ul ¿IOCH IVIA / a / ZUZZ / UI 6. The computer-implemented method according to claim 5, characterized in that the image includes a plurality of marriage advertisements captured from a newspaper.

7. The computer-implemented method according to claim 1, characterized in that the machine learning model includes a bidirectional short-term and long-term memory (LSTM) layer.

8. The computer-implemented method according to claim 1, characterized in that the computation of token representations for the set of tokens includes at least one of: computing an ELMo embedding for each of the set of tokens using a trained ELMo model; or computing a GloVe embedding for each of the set of tokens using a trained GloVe mode.

9. A computer-readable hardware storage device characterized in that it comprises instructions that, when executed by one or more processors, cause the one or more processors to perform operations to segment an input text, wherein the operations comprise: extracting a set of tokens from the input text; computing token representations for the set of tokens; providing the token representations to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided with input token representations, and wherein each of the set of label predictions indicates a position of a particular token from the set of tokens with respect to a particular segment;and determine one or more segments within the input text based on the set of label predictions.

10. The computer-readable hardware storage device according to claim 9, characterized in that the operations further comprise: receiving an image; and generating the input text based on the image using a character recognition module.

11. The computer-readable hardware storage device according to claim 10, characterized in that the calculation of token representations for the set of tokens includes: calculating a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image.

12. The computer-readable hardware storage device according to claim 10, characterized in that the character recognition module is an optical character reader.

13. The computer-readable hardware storage device according to claim 12, characterized in that the position of the particular token with respect to the particular segment is one of: at the beginning of the particular segment; within the particular segment; or outside the particular segment.

14. The computer-readable hardware storage device according to claim 13, characterized in that the image includes a plurality of marriage announcements captured from a newspaper.

15. The computer-readable hardware storage device according to claim 9, characterized in that the machine learning model includes a bidirectional short-term and long-term memory (LSTM) layer.

16. The computer-readable hardware storage device according to claim 9, characterized in that the computation of token representations for the set of tokens includes at least one of: calculating an ELMo embedding for each of the set of tokens using a trained ELMo model; or calculating a GloVe embedding for each of the set of tokens using a trained GloVe mode.

17. A system for segmenting an input text, the system being characterized in that it comprises: one or more processors; and a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: extracting a set of tokens from the input text; computing token representations for the set of tokens; providing the token representations to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided with input token representations, and wherein each of the set of label predictions indicates a position of a particular token from the set of tokens with respect to a particular segment;and determine one or more segments within the input text based on the set of label predictions.

18. The system according to claim 17, characterized in that the operations further comprise: receiving an image; and generating the input text based on the image using a character recognition module.

19. The system according to claim 18, characterized in that the calculation of token representations for the set of tokens includes: calculating a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image.

20. The system according to claim 18, characterized in that the position of the particular token with respect to the particular segment is one of: at the beginning of the particular segment; within the particular segment; or outside the particular segment.

Images