Task Detection in Communication Using Domain Adaptation

Abstract

The present invention generally discusses devices, systems and methods for task classification. A method may include: modifying a representation of a source sentence of a source sample from a source corpus to more closely resemble a representation of a target sentence of a target sample from a target corpus, using a machine learning model to operate with the target sample to generate a task tag, the machine learning model being trained using the modified representation of the source sentence, the task tag indicating whether the target sample includes a task, and causing a personal information manager (PIM) to generate a reminder based on whether the target sample includes the task.

Description

Background Art

[0001] Over the past few decades, the use of email has increased both in terms of total volume (information overload) and the number of purposes for which email is used (functionality overload). Many email programs serve as communication tools as well as task repositories, personal archives, and calendaring tools. While various studies have focused on how to support this functionality overload by directly integrating task support or calendaring within email clients, few studies have focused on how information residing in other data sources can be used to alleviate information overload. Additionally, few studies have focused on the search problems that exist when searching or re-finding items on a personal information manager (PIM) (e.g., email) repository.

[0002] Prior research includes work on automated email management and task progress monitoring. Prior research on commitment detection and domain adaptation includes showing that annotation of commitments and requests in emails is challenging, even for humans. A commitment is a promise that an individual promises to perform an action, and a request is a solicitation for a recipient to perform the requested action. At least one research topic even developed guidelines for collecting judgments and building a dataset for commitment / request classification. One insight from this research is that the annotation task is easier and more accurate when the statements are given in context (the full email). Other prior research suggests fine-grained classification of commitments in communications and studies which phrases are associated with commitments.

[0003] However, automatic commitment classifiers have been developed in other works. For example, some studies have proposed training a classifier to classify sentences in emails as one of the following actions: delivery, commitment, request, correction, proposal, and meeting. In this study, sentences are represented by term frequency-inverse document frequency (TF-IDF) weighted vectors on n-grams.

[0004] Other studies have used more complex features to detect promises in emails, such as named entities, part-of-speech (POS) tags, relevance, and coreference resolution. This study considers both requests and promises as promises.

[0005] There are other studies that use different kinds of features to detect tasks in emails. This study considers the promise as one of the tasks extracted from emails, and considers both requests and promises as promises. This study uses a feature set such as message length, presence of modal verbs, and question words, and trains a classifier on a manually labeled set of emails. This study concludes that only some regions of the email are relevant for promise detection, and other regions generally introduce noise.

[0006] All of these studies used a single dataset to train the models and perform the analysis. The datasets used in these studies were also small (e.g., less than 10,000 examples used to train the models). Although one study performed limited analysis on the transferability of the commitment model, that study used a dataset from the same domain.

[0007] For privacy reasons, it may be impractical to train commitment detection models (and other email-based models) using proprietary data. Instead, these models are trained based on a public corpus. The problem with training based on this public data is that the resulting model is biased with respect to the private / proprietary target audience, resulting in suboptimal performance on the target. An embodiment provides a method to reduce or remove the bias from models trained using such a public corpus. An embodiment can provide models that perform well on proprietary targets without requiring or requiring only minimal inspection of the target corpus. Summary of the invention

[0008] This Summary section is provided to introduce various aspects of the embodiments in a simplified form, and the following embodiments are further explained in the Detailed Description. This Summary section is not intended to identify necessary or required features of the claimed subject matter, and the combination and order of elements listed in this Summary section are not intended to provide limitations to the elements of the claimed subject matter.

[0009] A system may include a memory. The system may also include a processing circuit system coupled to the memory to implement operations for task classification of a target sample from a target corpus. Task classification may include: modifying the representation of a source sentence of a source sample from a source corpus to more closely resemble a target sentence of a target sample from a target corpus. Task classification may also include: using a machine learning model to operate with the target sample to generate a task tag, the machine learning model being trained using the modified representation of the source sentence, the task tag indicating whether the target sample includes a task. The processing circuit system may also cause a personal information manager (PIM) to generate a reminder based on whether the target sample includes a task.

[0010] At least one machine-readable storage medium may include instructions for execution by a processing circuit system to perform operations for task detection. The operations may include: modifying a representation of a source sentence of a source sample from a source corpus to more closely resemble a representation of a target sentence of a target sample from a target corpus. The operations may also include: operating with the target sample using a machine learning model trained using the modified representation of the source sentence to generate a task tag indicating whether the target sample includes a task. The operations may also include causing a personal information manager (PIM) to generate a reminder based on whether the target sample includes a task.

[0011] A method performed by at least one processor of a computing system may include: modifying a representation of a source sentence of a source sample from a source corpus to more closely resemble a representation of a target sentence of a target sample from a target corpus. The method may also include: operating with the target sample using a machine learning model trained using the modified representation of the source sentence to generate a task tag indicating whether the target sample includes a task. The method may also include causing a personal information manager (PIM) to generate a reminder based on whether the target sample includes the task. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A diagram illustrating an embodiment of an email interface is provided by way of example.

[0013] Figure 2 A diagram of an embodiment of a system for domain-independent classification of tasks is illustrated by way of example.

[0014] Figure 3 A diagram of another embodiment of a system for domain-independent classification of tasks is illustrated by way of example.

[0015] Figure 4 A diagram of another embodiment of a system for domain-independent classification of commitments is illustrated by way of example.

[0016] Figure 5 An exploded diagram of an embodiment of an autoencoder is illustrated by way of example.

[0017] Figure 6 An exploded view of an embodiment of a sequence-to-sequence encoder is illustrated by way of example.

[0018] Figure 7 A diagram illustrating an embodiment of a method for task classification is provided by way of example.

[0019] Figure 8 A block diagram of an embodiment of a machine (eg, a computer system) for implementing one or more embodiments is illustrated by way of example. DETAILED DESCRIPTION

[0020] In the following description, reference is made to the accompanying drawings that form a part thereof and that show by way of illustration specific embodiments that can be put into practice. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It is to be understood that other embodiments may be used, and that structural, logical and / or electrical changes may be made without departing from the scope of the embodiments. Therefore, the following description of the embodiments is not intended to be limiting, and the scope of the embodiments is limited by the appended claims.

[0021] In some embodiments, the operations, functions or algorithms described herein may be implemented in software. Software may include computer executable instructions stored on a computer or other machine-readable medium or storage medium, such as one or more non-transient memories (e.g., non-transient machine-readable media) or other types of hardware-based storage devices (local or networked). Further, such functions may correspond to a subsystem, which may be software, hardware, firmware, or a combination thereof. Multiple functions may be performed as needed in one or more subsystems, and the described embodiments are examples only. Software may be executed on a digital signal processor, an ASIC, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or other types of processors, which operate on a computer system (such as a personal computer, a server, or other computer system) to convert such a computer system into a specially programmed machine. The functions or algorithms may be implemented using processing circuit systems, such as those that may include electrical and / or electronic components (e.g., one or more transistors, resistors, capacitors, inductors, amplifiers, modulators, demodulators, antennas, radios, regulators, diodes, oscillators, multiplexers, logic gates, buffers, caches, memories, GPUs, CPUs, field programmable gate arrays (FPGAs), etc.).

[0022] Email has been a key web-based communications medium for more than 25 years. Although email was originally designed for asynchronous communication, people have "overloaded" the use of email with other functions, such as task management and personal archiving. As online services and the web have grown, email has not only continued to serve these purposes, but in an increasing number (e.g., as a receipt filing cabinet for e-commerce purchases, as a standard part of identification / authentication flows, and for calendar management).

[0023] Regarding calendar management, since meeting arrangements and time negotiations are often conducted via email, almost every modern email service (both web email and client applications) provides full-featured calendar management. Despite this integration, most feature development has focused on moving information from email to the user's calendar, but little work has focused on the impact of calendar information on improving other functionality of personal information management (PIM) services.

[0024] People use email not only as a communication tool, but also for creating and managing tasks and managing calendars. When the number of ongoing tasks created via email increases, people may struggle to manage their tasks and monitor their progress. Automatic task management systems (such as PIM) can overcome this problem and help people manage their tasks more effectively. Commitments such as "I will send a report by the end of today" are a type of task involving the completion of future actions between individuals. Another task type is a request. A request includes asking another person to perform an action. The task detected in this article is part of a communication between a user of a first device and a user of a second device. The task can be part of a text, email, instant messaging communication, file, attachment, etc.

[0025] Figure 1 A diagram of an embodiment of a task 114 in an email 100 presented on a user interface is illustrated by way of example. The email 100 may be part of a data domain (sometimes referred to as a corpus) in XML, HTML, or other formats, where metadata describes the content of the email 100. The illustrated email 100 includes a from: line 102, a to: line 104, a subject 106, a time 108, (multiple) attachments 110, and a body 112. The from: line 102 indicates who the email 100 originated from. The to: line 104 indicates who the email 100 is sent to. In many emails, the to: line includes a carbon copy contact (cc:) and a separate blind carbon copy contact (bcc:) line. The subject 106 typically summarizes the content to be covered in the email 100. The time 108 is typically associated with a time zone and indicates the date and time of the email 100. The (multiple) attachments 110 sender indicates in the from: line 102 the resources that are considered relevant to the email 100. The body 112 includes text, images, and other data that describes in more detail the reason for the email 100. Any information of the email 100 can be used to help identify the task performed in the email 100. For example, the body 112 in the email 100 includes the task 114 "I will forward to entity X...".

[0026] Such tasks are often expressed in emails or other communications (such as direct messages, calendar events, etc.), and users may struggle to review and complete them in a timely manner. Detecting tasks (e.g., automatically, such as after deployment or installation without human intervention) and providing reminders of the tasks enables task management tools such as personal information managers (PIMs) or digital assistants to generate reminders or notifications to help users fulfill their obligations. Despite the potential benefits of automatic task detection, work in this important area has been limited to only a small number of studies.

[0027] A commitment involves a promise made between individuals to complete a future action. Detecting commitments made in communications (e.g., "I will send the report by the end of today") enables the PIM to help users review the commitments they made and help users complete the commitments in a timely manner. Embodiments herein may reliably detect commitments or other tasks in emails, instant messages, text messages, files, attachments, phone or meeting notes, teamwork tools, or other communications. Embodiments may use one or more machine learning models trained on a first domain (e.g., a corpus of communications generated in a first context) and implemented to tag sentences from communications in a second domain (e.g., a corpus of communications generated in a second context).

[0028] Previous commitment detectors achieve degraded performance when the test domain differs from the training domain. It is not always practical to train a detector directly using samples from the test domain. This may be due to the unavailability of test data at training time or because access to test data is restricted for privacy reasons. This illustrates the domain bias associated with communication datasets and the need for more robust and generalizable models for task detection and task extraction in general.

[0029] Embodiments herein can learn domain-independent task models, such as reducing the reduction achieved when the test domain is different from the training domain. In an embodiment, the differences between domains (communication corpora) are identified and used to transfer knowledge between domains. This knowledge transfer can be at different levels of granularity: feature-level adaptation, sample-level adaptation, or a combination thereof. Other embodiments can use neural autoencoders, such as those trained in an adversarial manner, to train samples to learn domain-independent representations. Transfer learning can help remove domain biases to obtain models with less domain relevance. Domain differences can have a significant negative impact on the quality of communication-based models. Through transfer learning, embodiments can help deal with this problem.

[0030] Task detection is challenging for at least two reasons. First, the task detection operation itself is inherently difficult, even for humans, and is particularly difficult when the text is short and has limited context. Second, communication models are trained and shipped to users based on potentially biased data sets. For privacy and practical reasons, communication-based models are usually derived from public data sets, which are biased in various ways. For example, a data set compiled by the Cognitive Assistance for Learning and Organizations (CALO) project called the "Enron Email Dataset" (referred to as "Enron" in this article) (mostly including emails from senior officials of a company called Enron) and a data set called the "Avocado Research Email Collection" (referred to as "Avocado" in this article) including emails and attachments from a now non-existent information technology (IT) company named Avocado are two commonly used email data sets to learn email models. These data sets belong to two organizations with different areas of focus and contain emails from different time periods. The terms including named entities and technical jargon vary greatly in domain and time. This means that a model learned on one dataset may be biased and may not perform well on a different target dataset at some point in the future. These biases may affect the models trained on these email corpora.

[0031] Transfer learning is a domain adaptation method that enables knowledge learned in one domain (source domain) to be transferred to another domain (target domain). Using this method to transfer knowledge learned from one communication set to another communication set can help achieve more robust and generalizable models for task detection.

[0032] Some methods of transfer learning are discussed herein. The context of this discussion is for task detection, however embodiments are also applicable to other contexts, such as image classification, text classification, security, etc. In various examples, sentences are extracted from text, such as from one or more of the following: an email, an attachment, a calendar event, a meeting, a direct messaging service, a text message, or other communication. Tags may be assigned to at least some sentences, indicating whether the sentence corresponds to a task.

[0033] A variety of techniques for transfer learning are presented in this paper. These include feature-level adaptation, sample-level adaptation, and transfer learning autoencoders. For feature-level adaptation, a mapping between features (e.g., n-grams) in the source domain and the target domain is identified, and the mapping is used to transfer the features of the source domain to the target domain and learn a classifier using the migrated data.

[0034] Given a collection of labeled samples from a source corpus “S”, the goal can be to create a model that has high task detection accuracy when applied to target samples from a target corpus “T”. Three different techniques and their variants for transferring classification knowledge between domains are described in Figures 2 to 5 Provided in: Feature-level adaptation ( Figure 3 ), sample-level adaptation ( Figure 2 ) and autoencoders using feature-level adaptation and sample-level adaptation ( Figure 4 and Figure 5 ).

[0035] Figure 2 A diagram of an embodiment of a system 200 for domain-independent classification of tasks is illustrated by way of example. The illustrated system 200 includes a source corpus 202, a target corpus 204, a sample adaptation program 206, a machine learning program 208, a model program 210, and a PIM 212.

[0036] Source corpus 202 includes source samples 203 (e.g., files, documents, packets, data streams, etc., such as email data, attachments, calendar event data, appointment data, digital annotation data, direct messaging data, text data, etc.). Source samples 203 may be created in a specific culture or geographic region, where the habits that indicate a task may have unique nuances that are not realized elsewhere. Therefore, source samples 203 may be generated using a first dictionary.

[0037] The target corpus 204 includes target samples 205 similar to the source samples 203, wherein the target samples 205 are generated using different conventions (e.g., using different dictionaries) indicating the task. The target samples 205 are to be classified by a model program 210, which is trained based on the source samples 203 or the adapted source samples 207.

[0038] Sample adaptation (the operation performed by sample adaptation program 206) is sometimes referred to as importance sampling. Importance sampling is a technique for estimating the parameters of a distribution (the distribution of target corpus 204) given samples generated from a different distribution (the distribution of source corpus 202). Assume there are two distributions: the P distribution from which the samples in source corpus 202 are generated. S (x, y) and the samples in the target corpus 204 are generated P T (x, y). In the setting of task detection operation, x is the sentence and y is its corresponding task label. Assume that the goal is to create a model using labeled samples from S while optimizing the objective (e.g., classifier loss) of samples in T. The importance sampling method for classification can operate as follows. As mentioned, the goal can be to find the parameters of the classifier that minimize the loss of samples in T, given samples from S,

[0039]

[0040] The assumption in the above derivation is that P T (y i |x i )=P S (y i |x i ), which implies the conditional probability of the class given a domain-independent sample. Often referred to as importance weights. Given the above derivation, a classifier can be trained using samples 203 from the source corpus 202, modified by its importance weights. S (x i ) and P T (x i ) are the marginal distributions of samples in the source corpus 202 and the target corpus 204, respectively. In order to find θ * , source sample 203 from source corpus 202 and P S (x i ) and P T (x i ) can be used.

[0041] To estimate P S (x i ) and P T (x i ), a set of source samples 203 from the source corpus 202 and a set of target samples 205 from the target corpus 204 may be used. The target samples 205 may be considered as the positive class, and the source samples 203 may be considered as the negative class. The domain classifier of the sample adaptation program 206 may be trained to predict for each sample the likelihood that the sample is generated from the target corpus 204. The LR model may be used to train the domain classifier. For each sample x i , P T (x i )=p and P S (x i ) = 1-p, where p is x i Assign to x from the trained domain classifier i The probability of generating the target corpus 204T.

[0042] The adapted source samples 207 (reweighted source samples) and corresponding source sample labels can be provided to a machine learning program 208. The machine learning model 208 can use statistical techniques that enable the computing device to "learn", more accurately, to infer the most likely results based on the determined statistical information. The machine learning model 208 may be particularly useful to solve problems that do not have obvious inspiration to solve the problem. Two types of machine learning programs include supervised learning and unsupervised learning. Supervised machine learning includes providing a model with example inputs and corresponding expected outputs. The weights of the model are updated so that when provided with the corresponding input, the expected output is generated by the model. The model can then be provided with the input, which has not been previously operated to predict which output the input corresponds to. In unsupervised learning, input and output examples are not provided to the model, and the model is entrusted to find the structure in the input alone. Unsupervised learning can be used to identify features of the input, which can be used to uniquely identify the input. The embodiments herein can apply supervised learning to the problem of extracting tasks from communications.

[0043] In some embodiments, the machine learning program 208 may include a neural network. A neural network (NN) is a computational structure closely modeled on biological neurons. Typically, a NN encodes information (e.g., data or decisions) via weighted connections (e.g., synapses) between nodes (e.g., neurons). Modern NNs are the basis of many machine learning applications.

[0044] Many NNs are represented as weight matrices corresponding to the connections being modeled. NNs operate by accepting data into a set of input neurons, which typically have many outgoing connections to other neurons. At each pass between neurons, the corresponding weights modify the inputs and are tested against a threshold at the destination neuron. If the weighted value exceeds the threshold, the value is weighted again, or transformed by a nonlinear function, and transmitted to another neuron further down the NN graph - if the threshold is not exceeded, then typically the value is not transmitted to the lower graph neuron, and the synaptic connection remains inactive. The process of weighting and testing continues until an output neuron is reached; the pattern and value of the output neuron constitutes the result of the NN processing.

[0045] The correct operation of most NNs depends on accurate weights. However, the NN designer usually does not know which weights will act on a given application. Instead, a training process is used to arrive at appropriate weights. The NN designer usually chooses multiple neuron layers or specific connections between layers that include circular connections. Instead, the training process usually continues by selecting initial weights, which can be randomly selected. The training data is fed into the NN and the results are compared to an objective function that provides an error indication. The error indication is a measure of how wrong the results of the NN are compared to the expected results. The error is then used to correct the weights. After many iterations, the weights will converge together to encode the operating data into the NN. This process can be referred to as optimization of an objective function (e.g., a cost or loss function), where the cost or loss is minimized.

[0046] Gradient descent techniques are often used to perform objective function optimization. Gradients (e.g., partial derivatives) are calculated relative to layer parameters (e.g., various aspects of weights) to provide a direction of correction and possibly a degree of correction, but do not result in a single correction to set the weights to the "correct" value. That is, over multiple iterations, the weights will move to the "correct" value or an operationally useful value. In some implementations, the amount of movement or step size is fixed (e.g., the same between iterations). Small step sizes tend to take a long time to converge, whereas large step sizes may oscillate around the correct value, or exhibit other undesirable behaviors. Variable step sizes can be attempted to provide faster convergence without the disadvantages of large step sizes.

[0047] Back propagation is a technique in which training data is fed forward through a NN, where "forward" means that the data starts at the input neuron and follows the directed graph of neuron connections until the output neuron is reached, and the objective function is applied backward through the NN to correct the synaptic weights. In each step of the back propagation process, the results of the previous step are used to correct the weights. Therefore, the results of the output neuron correction are applied to the neurons connected to the output neuron, and so on, until the input neuron is reached. Back propagation has become a popular technique for training various NNs. Any well-known optimization algorithm for back propagation can be used, such as stochastic gradient descent (SGD), Adam, etc.

[0048] For machine learning of the adapted source samples 207 (and labels), supervised techniques may be employed. Importance sampling may be provided by the machine learning program 208 and is used to weight the training samples (adapted source samples 207) of the source corpus 202 based on the similarity of the target samples 205 of the target corpus 204. The weighted samples in the source corpus 202 may be used to learn a classifier through the machine learning program 208. Specific machine learning techniques that may be implemented by the model program 210 using weights 209 include decision tree learning, association rule learning, NN, deep learning, inductive logic programming, support vector machines, clustering, Bayesian networks, reinforcement learning, representation learning, similarity and metric learning, sparse dictionary learning, genetic techniques, rule-based machine learning, and learning classifier systems.

[0049] The model program 210 can implement a classifier. The model program 210 can receive the target sample 205 and generate an output indicating whether the target sample 205 corresponds to a task 211. The machine learning program 208 can learn weights 209 to be applied to the model to form a classifier to be implemented by the model program 210. Features of the samples 203, 205 indicating the task include, for example, "I will", "I", "will", "I will", "let you know", "let you", "call you", "I should", "we will", "will call", etc.

[0050] The model program 210 in question may generate an indication of whether the target sample 205 corresponds to a task 211. The task 211 may include one or more of the following: a binary value indicating whether the target sample 205 corresponds to a task (e.g., a logical "1" may indicate that the target sample 205 corresponds to a task, and a logical "0" may indicate that the target sample 205 does not correspond to a task, and vice versa); a value indicating the type of task to which the target sample 205 corresponds (e.g., a promise to perform an action, such as setting up a meeting, attending a meeting, walking the dog, sending flowers, sending an email or document, paying a debt, contacting another person not included in the target sample 205, mowing the lawn, or any other action); a time corresponding to the task 211; a time to remind the user of the task 211, etc. Alternatively, the PIM 212 may receive the target sample 205 in which the task is detected, and may determine the type of task, the time corresponding to the task 211, the time to remind the user of the task 211, etc.

[0051] The PIM 212 may include a calendar interface 216, an email interface 214, and a task interface 218. Examples of PIMs include Outlook from Microsoft Corporation of Redmond, Washington, United States of America (USA), Gmail from Google of Menlo Park, California, USA, EssentialPIM from Astonsoft Ltd. of Estonia, Amazon Simple Email Service (SES) from Amazon.com Inc. of Seattle, Washington, USA, Thunderbird from the Mozilla Foundation of Mountain View, California, USA, eM Client from eM Client Inc. of San Francisco, California, USA, and Zimbra Desktop from Synacor Corporation of Buffalo, New York, USA, among others. Not all PIMs include the email interface 214, calendar interface 216, and task interface 218 functionality. An embodiment may be integrated with one or more separate calendar, email, or task PIMs such that the PIM 212 need not include all of the email, task, and calendar functionality.

[0052] The PIM application 212, with or without user intervention, can manage appointments, meetings, tasks, and / or reminders through the calendar interface 216. The calendar interface 216 provides an interface in which data about dates, events, attendees, subjects, attachments, descriptions of events (e.g., text), reminder times, etc. can be recorded. There are many configurations of calendar interfaces, and embodiments are not limited to a particular calendar configuration.

[0053] The PIM application 212, with or without user intervention, can manage emails through an email interface 214. The email interface 214 provides an interface for organizing, accessing, searching, storing, deleting, or performing other operations on emails. There are many configurations of email interfaces, and embodiments are not limited to a particular email configuration.

[0054] The PIM application 212, with or without user intervention, can manage tasks through a task interface 218. The task interface 218 provides an interface for organizing, accessing, searching, storing, deleting, editing, or performing other operations on tasks to be performed. There are many configurations of task interfaces, and embodiments are not limited to a particular email configuration.

[0055] After receiving the task 211 (or determining how to handle the task 211), the PIM 212 can automatically (without user intervention) schedule a meeting, appointment, etc. to be presented to the user through the calendar interface 216, draft an email to be presented to the user through the email interface 214, or generate a task to be presented to the user through the task interface 218. The interface (e.g., the email interface 214, the calendar interface 216, or the task interface 218) can present the user with an explanation as to why the email, calendar event, or task was created. For example, the interface can display a software control with text that explains "You promised to send this email to entity X in another email on date Y. If you want to fulfill the task, please edit and select 'Send', select 'Remind me later' to be reminded at a later time, or otherwise select 'Cancel' to terminate the task." Through this interface, the user can therefore set a reminder, create a task, send an email, create an appointment or meeting, etc. based on the task 211.

[0056] In some embodiments, due to the task 211 in the target sample 205, the user may have scheduled a meeting, sent an email, created a task, etc. The PIM 212 may search the user's emails, tasks, and calendar events to determine whether the user has already set up or sent such a reminder, email, task, or calendar event. If such a reminder, email, task, calendar event, or other task 213 has already been set up or sent, the PIM 212 may take no action, and if such a reminder, email, task, or other task 213 has not yet been set up or sent, the task 213 may be automatically generated.

[0057] In some embodiments, a user may provide data to PIM 212, such as through an email interface 214, a calendar interface 216, a task interface 218, a pop-up box, or other interface. The data provided by the user may be feedback data 220. Feedback data 220 may include whether sample 205 includes a task. Feedback data 220 from the user, predictions from model 210 (task 211), target sample 205, or other data may be provided to machine learning program 208. Feedback data 220 may be used to update input-output examples, or otherwise update data to train model program 210.

[0058] Figure 3 A diagram of another embodiment of a system 300 for domain-independent classification of tasks is illustrated by way of example. The system 300 is similar to the system 200, wherein the system 300 includes a feature adaptation procedure 306 instead of the sample adaptation procedure 206. It is noted that Figure 3 Some projects have Figure 2This indicates that the item may perform a different operation or have a different Figure 2 The projects with the same name have different structures. However, the general description of the projects with the same name is similar, except for Figure 3 Described differently.

[0059] The feature adaptation program 306 receives the source sample 203 and the target sample 205 and generates an adapted source sample 307. The adapted source sample 307 includes a representation of the source sample 203 that is modified to be more similar to the representation of the target sample 205. The modification may include word replacement, replacement of a representation of a word with a representation of another word, etc. For example, consider an embodiment where the source corpus 202 includes emails from a gas company and the target corpus 204 includes emails from an electric company. The feature adaptation program 306 may learn or otherwise be programmed to replace the representation of the term "gas" in the source sample 203 with the term "electricity", etc., to generate the adapted source sample 307. Other replacements may include people in one company using abbreviations such as "OOO" (out of office) to mean unavailable, and people in another company using "unavailable", "on vacation", etc. to mean the same thing. These differences may appear in emails or other communications. A model trained without knowledge of these differences may be less accurate in classifying the target sample 205 than a model trained with knowledge of these differences. Many other examples of these differences exist and are within the scope of the present disclosure.

[0060] The feature adaptation program 306 learns the features of the source sample 203 and the target sample 205. In some embodiments, the feature adaptation program 306 can determine a mapping between the learned features of the source sample 203 and the target sample 205. Assume that F s i is a feature set of samples from the source corpus 202, and is a feature set of samples from the target corpus 204. The feature adaptation procedure 306 can learn the mapping as follows in is to approximate the feature F s i Transform to Features The adapted source samples 307 may include the learned transform Modified source sample 203.

[0061] Features of the target sample 205 or source sample 203 may include one or more words, phrases, sentences, symbols, characters, images, combinations thereof, parts thereof, and the like. Features extracted from the source sample 203 or target sample 205 may indicate a task or help determine whether a task exists in the sample. For example, features of the source sample 203 may include the characters "CYL". These characters may indicate a task of calling someone later ("call you later"). The feature adaptation program 306 may replace one or more values ​​indicating "CYL" in the source sample 203 with values ​​indicating the phrase "call you later" to create an adapted source sample 307. Another example includes replacing all values ​​indicating the word "electricity" with one or more values ​​indicating the word "gas" or "oil", and the like. This replacement may help the machine learning program 308 create weights that take into account the differences in common terms in the source corpus 202 and the target corpus 204. These are merely examples of feature adaptation and corresponding learning, which are intended to provide a deeper understanding of the operation of the feature adaptation program 306, the adapted source sample 307, the machine learning model 308, and the model weights 309.

[0062] More generally, feature-level adaptation techniques such as those implemented by feature adaptation program 306 attempt to remove domain bias from features by removing domain-specific features or transforming features from a source corpus dictionary to a target corpus dictionary. Learning mappings between features from different domains has been studied in machine translation. Here, by contrast, the domains are two different dictionaries. Feature adaptation program 306 can implement one or more of two feature-level adaptation techniques: feature selection and feature mapping. Feature mapping has been discussed previously.

[0063] Feature selection involves detecting domain-specific features in the source corpus 202 and the target corpus 204 and removing them from the training and test samples. In feature selection, assume that there is a set D of unlabeled source samples 203 in the source corpus 202. S and the set D of unlabeled target samples 205 in the target corpus 204 T. In order to remove domain-specific features from samples 203, 205, the domain classifier of the feature adaptation program 306 can be trained as a "positive" sample using the source sample 203 and trained as a "negative" sample using the target sample 205. The classifier can be trained to distinguish between the source sample 203 and the target sample 205. The classifier can identify the most distinguishing (informative) features of the samples 203, 205. Logistic regression (LR) or another machine learning technique for classification can be used to train the classifier. The top K features from the classification model can be recorded. The top K features of the source sample 203 can be replaced with unique symbols before training. Similar procedures can be used to remove domain-specific words from the target sample 205 of the target corpus 204. After removing domain-specific information from the source sample 203 and the target sample 205, the task classifier can be trained on the adapted source sample 307, such as by the machine learning program 308. The task classifier, model program 310 can be used to predict task tags on the target sample 205 from the target corpus 204. A task classifier is an example of a model program 310 .

[0064] Feature mapping identifies equivalent features between the source domain and the target domain, and transforms features from samples 203, 205 to their equivalents in other samples 205, 203 before training the task classifier (model program 310). Features can be n-grams (e.g., 1≤n≤3) in some embodiments. Assumptions of the feature mapping technique can include that for each feature in the source (target) domain, there is an equivalent feature in the target (source) domain. Therefore, for each domain-specific feature in the source corpus 202, there is a domain-specific equivalent in the target corpus 204, and the goal can be to find these equivalents.

[0065] An embodiment of the feature adaptation procedure 306 is now described in more detail. Assume that there is a set of samples 203, 205 in each domain. Sentences can be extracted from the samples 203, 205. For each domain, a semantic space can be learned, where each feature is represented as a low-dimensional neural embedding vector. In some embodiments, the cross-word sequence architecture of Word2Vec (a two-layer NN from Mikolov et al.) can generate embeddings for words. A linear transformation between the embedding spaces of the source samples 203 and the target samples 205 can be determined. The linear transformation Can be used to transform features between domains. Word embedding is a natural language processing technique in which one or more words, phrases, symbols, characters, etc. are mapped to corresponding vectors of real numbers. Word embedding is a kind of compression because it takes input in a one-dimensional space for each word and maps each word to a lower-dimensional vector space. The mapping can be generated using NN, dimensionality reduction on word co-occurrence matrices, probabilistic models, interpretable knowledge base methods, and explicit representations of the context in which words appear.

[0066] Transformations based on linear mappings of embeddings between spaces have been used to transform and detect semantic drift. In an embodiment, a set of words can be used as anchors between domains and training samples to learn mappings. Stop words (such as "reach", "the", "...", "...", "...", "and", "or", etc.) can be used as anchors. Stop words should have the same meaning in both domains, and they can be used as fixed points from which features with varying meanings (usages) can be learned. Using training samples, the goal can be to learn a transformation matrix W from domain S to domain T ST , which minimizes the distance between words and their mapping vectors.

[0067] The objective function of such a function to be minimized by learning may include:

[0068]

[0069] The sum can take over the training features. A standard stop word list can be used, such as with a few additional words added (e.g., very frequent words in corpus 202 or 204) to learn the transformation matrix W ST . Gradient descent techniques can be used to optimize the objective function. and is the feature embedding w in the embedding space created for the source corpus 202 and the target corpus 204, respectively. Using the learned transformation, the feature w from the source corpus 202 S The mapping in the target corpus 204 may be determined as follows:

[0070]

[0071] where cos is the cosine similarity, and F T It is the set of all features (n-ary) in the target domain T.

[0072] The machine learning program 308 is similar to the machine learning program 208, but the machine learning program 308 operates on different inputs, namely, the source samples 307 to which the features are adapted instead of the source samples 207 to which the samples are adapted. The machine learning program 308 is therefore very likely to generate weights 309 that are different from the machine learning program 208. Similarly, the model program 310 may be different from the model program 210 because the weights 309 may be different from the weights 209. Although the purpose of the model program 310 is the same as that of the model program 210. That is, to identify whether the target sample 205 includes a task and to generate a task 311. The task 311 may include similar information as the task 211. The PIM 212 may receive the task 311 and generate a task reminder 313. The task reminder 313 differs from the task reminder 213 only in that it is generated based on different inputs, namely, the task 311 instead of the task 211.

[0073] Figure 4 A diagram of another embodiment of a system 400 for domain-independent classification of tasks is illustrated by way of example. System 400 is similar to systems 200 and 300, wherein system 400 includes an autoencoder program 406 instead of sample adaptation program 206 and machine learning program 208 or feature adaptation program 306 and machine learning program 308. Note that, Figure 4 Some projects have Figure 2 and 3 This indicates that the item may perform a different operation or have a different Figure 2 or 3 projects of different structures with the same name. However, the general description of the projects with the same name is similar, except for Figure 4 Described differently.

[0074] The autoencoder program 406 learns the weights 409 of the model program 410 that minimize the cost function. The autoencoder program 406 may include at least one NN configured to reduce the reconstruction loss and the classification loss while increasing the domain classification loss so that the model program 410 cannot distinguish which corpus 202 or 204 the sample comes from after autoencoding.

[0075] Model program 410 is likely to be different from model programs 210 and 310, at least because weights 409 are likely to be different from model programs 210 and 310 because they are generated via different training processes. Although the purpose of model program 410 can be the same as model programs 210 and 310. That is, to identify whether target sample 205 includes a task, including generating task 411. Task 411 can include information similar to tasks 211 and 311 (with or without task tags). Task 411 is different from tasks 211 and 311 because it is generated by a different model program 410, so the results of model program 410 can be different. PIM 212 can receive task 411 and generate task reminder 413. Task reminder 413 is different from task reminders 213 and 313 only in that it is generated based on different inputs, that is, task reminder 413 instead of task reminder 213 or 313.

[0076] The autoencoder program 406 can be trained in an adversarial manner, such as to maximize task classification accuracy while minimizing reconstruction loss and minimizing corpus classification accuracy. In order to remove domain-specific information from the sample representation, a domain classifier program 415 can be added to the autoencoder. In some embodiments, the domain classifier program 415 can be implemented using a NN. Given a training sample from a source corpus 202 or a target corpus 204 (e.g., a source sample 203 or a target sample 205), the domain classifier 415 can attempt to predict the domain to which the sample 203 or 205 belongs (e.g., the source corpus 202 or the target corpus 204). The domain classifier 415 can be trained in an adversarial setting, wherein during training, its accuracy can be minimized. By minimizing the domain classifier accuracy, domain-specific information can be removed from samples 203, 205.

[0077] When the task detection model is applied across domains, the quality of task detection is significantly reduced. That is, if the model program 410 is generated based on the weights learned from the source sample 203 of the source corpus 202, the prediction of the task in the target sample 205 from the target corpus 204 is inaccurate. As previously discussed, different methods for characterizing the differences between the communication corpus and domain adaptation (specifically, transfer learning) can remove domain-specific biases from the task detection model. The domain adaptation methods discussed herein provide significant gains in the accuracy and recall of the task detection model.

[0078] The purpose of using the autoencoder procedure 406 can be to learn a domain-independent representation using a combined set of samples from the source domain and the target domain. After creating the representation, a classifier (model procedure 410) can be trained on the debiased representation (the representation produced by the autoencoder procedure 406). Sample-level adaptation and feature-level adaptation can be combined using the autoencoder procedure 406.

[0079] This embodiment can remove domain bias by introducing a domain classifier 415 that can be trained in an adversarial manner.

[0080] Embodiments herein may apply adversarial training to domain adaptation. Embodiments may include adding a domain classification loss to the loss of the task and attempting to maximize the domain loss. Maximizing the domain loss can help ensure that the learned representation does not contain domain information. Similarly, embodiments may use adversarial training to learn representations of sentences. However, embodiments may extend this work by using a sequence-to-sequence autoencoder for the model to learn representations.

[0081] Figure 5 An exploded view of an embodiment of an autoencoder program 406 is illustrated by way of example. The illustrated autoencoder program 406 includes an input program 550, a sample representation program 558, an output program 562, and a loss function 564.

[0082] Deep autoencoders have been shown to be successful in unsupervised feature extraction and representation learning. These models are unsupervised and model the underlying distribution from which the data is generated. In a transfer learning setting, autoencoders can be used to learn representations of a combined set of samples from the source corpus 202 and the target corpus 204. Using samples from the source corpus 202 and the target corpus 204, the autoencoder program 406 can learn a model that represents samples from the corpora 202 and 204. The learned representations produced by the model program 410 can be domain-independent.

[0083] After learning the representation, the representation of the samples in the source corpus 202 (with its labels) can be used to train the classifier. Note that the method can use a set of labeled samples from the samples in the source corpus 202 and a set of unlabeled samples from the target corpus 204.

[0084] The task detection problem is presented next, and then the autoencoder model of the embodiment is described. Then, the project details of the autoencoder program 406 including the input program 550, the sample representation program 558, the output program 562 and the loss function 564 are described.

[0085] Assume that the task sentence set from the source corpus 202 and the task sentence set from the target corpus 204 is available.

[0086] The autoencoder embedding 556 may be generated using an embedding projection procedure 554 that is trained using unlabeled source samples 203 from the source corpus 202 and target samples 205 from the target corpus 204. In some embodiments, the embedding projection procedure 554 may include a NN. In addition to the source samples 203, a set of labeled target samples 205 is applied to train the sample representation and output procedure 562. S and X T It can be used to create a corpus classifier 563 that has good task detection performance on source samples 203 from the source corpus 202 and target samples 205 from the target corpus 204 (eg, approximately the same performance on the source samples 203 and the target samples 205 ).

[0087] In general, the autoencoder program 406 receives a candidate sentence (e.g., sample 549) and predicts the likelihood that the sentence constitutes a task (e.g., between an email sender and an email recipient). The illustrated autoencoder program 406 includes three main components, namely, an input program 550, which provides a set of functions to convert each input sentence x into i Encoded as a sequence of dense vectors in corresponds to the embedding of the jth word in the sentence, and n is the number of words in the sentence. For simplicity, it can be assumed that d is the dimension of the representation vector of the word.

[0088] Sample representation program 558 is provided with embedding 556 (sequence of embeddings 556) of input program 550. Sample representation program 558 learns a representation of input sentence 552. The output of sample representation program can be a vector It is an aggregate representation of the input sentence 552. Further details of the sample representation procedure 558 are provided elsewhere herein.

[0089] The output procedure 562 determines the likelihood that the input sentence 552 includes a task based on the representation of the sentence 552 provided by the sequence-to-sequence encoder 560 .

[0090] Input program 550 can use at least one trainable embedding program 554 to project input sentence 552 into a set of sequential dense vectors of dimension d. Each vector in input 552 can correspond to a word.

[0091] Figure 6An exploded view of an embodiment of a sequence-to-sequence encoder 560 is illustrated by way of example. To represent a sample, the sample representation program 558 may include a sequence-to-sequence encoder 560. The sequence-to-sequence encoder 560 may include a recurrent neural network (RNN) or other NN. The sequence-to-sequence encoder 560 receives the embedding 556 in a "left to right" direction in a forward pass, (Embedding in embedding 556). Sequence-to-sequence encoder 560 may create a sequence of hidden states 670A to 670C, in is a dynamic function such as a long short-term memory (LSTM) or a gated recurrent unit (GRU) can be used. The backward pass of the sequence-to-sequence encoder 560 reads Z in the opposite direction. i Embed 556, that is, This generates a hidden state sequence 672A to 672C The concatenation of the last hidden state of the forward pass and the first hidden state of the backward pass of the sequence-to-sequence encoder 560 (Hidden states 672A, 670C) may be used as the final representation of embedding 556 provided to output program 562.

[0092] Output program 562 may include three output programs: decoder 577, task classifier 565, and corpus classifier 563. Decoder 577 may include a NN (e.g., an RNN) that attempts to generate input sequence 549 given a learned representation of the input sequence (the output of encoder 560). The purpose of decoder 577 may be to estimate the probability in is the output 579 of the decoder 577 at time step t. Decoding by the decoder 577 may be started by reading a special symbol (e.g., 'GO') at the first time step. Decoding may be stopped by reading another special symbol (e.g., 'EOS') at the end of each input sentence 549 by the decoder 577. Given the output at each time step, the output 579 may be determined as:

[0093]

[0094] The decoder 577 can be trained in an end-to-end training process. The training process can use a sample batch B = <x i ,y i , d i > to execute, where y i and d i They are samples x i Given an input sequence x i , the correct output is o iThe conditional log-likelihood of can be determined:

[0095]

[0096] Where N is the number of training samples.

[0097] The task classifier 565 may include a feed-forward NN layer with a nonlinearity 573 (e.g., hyperbolic tangent (tanh)) followed by a sigmoid 575. The task classifier 565 receives the learned representation of the input sequence (the concatenation of hidden states 672A and 670C) from the sample representation program 558 and predicts the probability of the sample 549 constituting a task:

[0098]

[0099] in and are the trainable projection matrix and bias, respectively, and d c is the size of the projection, and is a trainable vector.

[0100] The task classifier 565 can be trained in an end-to-end training process, where the data <x i ,y i ,d i > is provided and is predicted. The classification loss function 583 can use cross entropy loss or other losses to calculate the loss. The entropy loss can be determined as:

[0101]

[0102] Similar to the task classifier 565, the corpus classifier 563 may include at least one feed-forward layer with a non-linear NN 569. The non-linear NN 569 may be followed by a sigmoid layer 571. The sigmoid layer 571 may provide the output of the corpus classifier 563. The corpus classifier 563 may predict the probability of a sample 549 originating from the target corpus 204:

[0103]

[0104] in and are the trainable projection matrix and bias, respectively, and d D is the size of the projection, and is a trainable vector.

[0105] The task classifier 565 can be trained in an end-to-end training process similar to the corpus classifier 563. <x i ,y i ,di > can be provided and can be predicted. The classifier loss function 583 can use cross entropy loss or other losses to calculate the loss. The entropy loss can be determined as:

[0106]

[0107] The goal of the training process may be to maximize the loss of the corpus classifier 563, such as to avoid capturing domain-specific information during learning sentence tokens. Given the described loss functions 581, 583, and 585, the final objective function determined by the loss procedure 587 of the autoencoder procedure 406 may be:

[0108] L=∝L R +βL C +γL D

[0109] Tasks 211, 311, 411 are any sentences in corpus 202, 204 that include a promise to perform an action or a request to perform an action that may be acted upon by PIM 212 (e.g., sending a document, sending an email, creating a meeting event, creating an appointment, adding a task to a task list, etc.) or worth reminding (e.g., meeting with a colleague). Task detection of an embodiment may be modeled as a binary classification task. Inputs such as source samples 203 or target samples 205 in a task detection task may include sentences, and outputs may include a binary tag indicating whether the sentence constitutes a task between entities.

[0110] Assume that there is a sentence set X extracted from the source corpus 202 = {x0, x1, ..., x N} and its associated task tag Y = {y0,y1,...,y N}, where x i is a sentence, and y i is annotated by x iAssigned binary task tags. X and Y can be used to train model programs 210, 310, 410, and then use model programs 210, 310, 410 to predict tags for tasks 211, 311, 411 of new sentences (such as target samples 205). Different representations can be used to represent source samples 203 in X: 1) n-gram bags: For feature-based models such as logistic regression (LR), n-grams (e.g., n>1) can be used, and samples 203, 205 can be represented by the frequency of n-grams therein; 2) n-gram part-of-speech (POS) bags: Similar to the previous representation, but with n-gram bags (e.g., n>1) of POS tags in samples 203, 205 instead of n-gram bags. Statistical Parsing and Language Analysis Toolkit (SPLAT) is a program from Microsoft Corporation of Redmond, Washington, USA, which can be used to extract POS tags. 3) Word sequence: For sequence-based models (such as autoencoders), samples 203, 205 can be represented as word sequences.

[0111] Figure 7 A diagram of an embodiment of a method 700 for performing task detection in a sample from a target corpus based on a sample from a source corpus is illustrated by way of example. The illustrated method 700 includes: in operation 710, modifying the source sample according to target sample characteristics; in operation 720, training a machine learning model based on the modified source sample; in operation 730, using the trained model to determine whether the target sample includes a task; in operation 740, providing a signal to the PIM based on the determination in operation 730; in operation 750, receiving feedback from a user as to whether the sample includes a task; and in operation 760, providing feedback data to further train the machine learning model. The source sample and the target sample may be corresponding sentences. The method 700 may also include: causing the PIM to generate: (i) a calendar event, (ii) a reminder, or (iii) an email based on whether the target sample includes a task.

[0112] Operation 710 may include performing feature adaptation to change features of the source sample to one or more specified values. Feature adaptation may include training a corpus classifier machine learning model to distinguish between the source sample and the target sample. Feature adaptation may include selecting the top K features from the corpus classifier machine learning model. Feature adaptation may include replacing each of the top K features in the target sample with the specified value.

[0113] Feature adaptation may include: identifying a mapping between features of the source sample and features of the target sample based on a distance between the words of the source sample and the corresponding mapping vectors. Feature adaptation may include: replacing representations of words of the source sample with mapping vectors in the corresponding mapping vectors.

[0114] Operation 710 may include performing importance sampling to change the weights of the source samples into importance weights. The importance weights may be determined based on the marginal distribution probabilities of samples classified as being from the source corpus and the marginal distribution probabilities of samples classified as being from the target corpus.

[0115] The machine learning model may include a NN autoencoder. Operation 710 may include: adversarially training the NN autoencoder to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss. The NN autoencoder may include a sample reconstruction NN that implements a sequence-to-sequence encoder that reads an embedded representation of a target sample in a first direction, creates a first hidden state sequence, reads the embedded representation of the target sample in a second direction opposite to the first direction, and creates a second hidden state sequence. The sample representation NN may provide a connection between the last hidden state of the embedded representation read in the first direction and the first hidden state of the embedded representation read in the second direction. The sample representation NN may include a recursive NN (RNN).

[0116] Figure 8 A block diagram of an embodiment of a machine 1000 (e.g., a computer system) for implementing one or more embodiments is illustrated by way of example. An example machine 1000 (in the form of a computer) may include a processing unit 1002, a memory 1003, a removable storage device 1010, and a non-removable storage device 1012. Although an example computing device is illustrated and described as machine 1000, the computing device may be in different forms in different embodiments. For example, the computing device may be a smartphone, a tablet computer, a smart watch, or a computer that includes information about the computer. Figure 8 Other computing devices having elements identical or similar to those illustrated and described. Devices such as smartphones, tablet computers, and smart watches are generally referred to as mobile devices. Further, although various data storage elements are illustrated as part of machine 1000, the storage device may also or alternatively include a cloud-based storage device accessible via a network (such as the Internet).

[0117] The memory 1003 may include volatile memory 1014 and non-volatile memory 1008. The machine 1000 may include or have access to a computing environment that includes various computer-readable media, such as the volatile memory 1014 and non-volatile memory 1008, a removable storage device 1010, and a non-removable storage device 1012. Computer storage devices include random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) and electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices capable of storing computer-readable instructions for execution to perform the functions described herein.

[0118] The machine 1000 may include or access a computing environment that includes input 1006, output 1004, and communication connection 1016. The output 1004 may include a display device that may also be used as an input device, such as a touch screen. The input 1006 may include one or more of the following: a touch screen, a touch pad, a mouse, a keyboard, a camera, one or more device-specific buttons, one or more sensors integrated into the machine 1000 or coupled to the machine 1000 via a wired or wireless data connection, and other input devices. The computer may operate in a networked environment using a communication connection to connect to one or more remote computers, such as a database server, including cloud-based servers and storage devices. The remote computer may include a personal computer (PC), a server, a router, a network PC, a peer device, or other public network node, etc. The communication connection may include a local area network (LAN), a wide area network (WAN), a cellular network, the Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), Bluetooth, or other networks.

[0119] Computer readable instructions stored on a computer readable storage device are executable by a processing unit 1002 of the machine 1000. Hard drives, CD-ROMs, and RAM are some examples of items that include non-transitory computer readable media such as storage devices. For example, the computer program 1018 may be used to cause the processing unit 1002 to perform one or more methods or algorithms described herein.

[0120] Additional notes and examples:

[0121] Example 1 includes a system comprising: a memory; a processing circuit system coupled to the memory to implement operations for task classification of a target sample from a target corpus, the task classification comprising: modifying a representation of a source sentence of a source sample from a source corpus to more closely resemble a target sentence of a target sample from the target corpus; using a machine learning model to operate with the target sample to generate a task tag, the machine learning model being trained using the modified representation of the source sentence, the task tag indicating whether the target sample includes the task; and causing a personal information manager (PIM) to generate a reminder based on whether the target sample includes the task.

[0122] In Example 2, Example 1 may further include, wherein modifying the representation of the source sentence of the source sample includes: performing feature adaptation to change a feature of the source sample to one or more specified values.

[0123] In Example 3, Example 2 may also include, wherein feature adaptation includes: training a corpus classifier machine learning model to distinguish between source samples and target samples; selecting top K features from the corpus classifier machine learning model; and replacing each of the top K features in the target sample with a specified value.

[0124] In Example 4, at least one of Examples 2 to 3 may further include, wherein modifying the representation of the source sentence of the source sample comprises: identifying a mapping between features of the source sample and features of the target sample based on a distance between a word of the source sample and a corresponding mapped target word of the target sample; and replacing the vector representation of the corresponding word of the source sample with a mapping vector of the corresponding mapped target word.

[0125] In Example 5, at least one of Examples 1 to 4 may further include, wherein modifying the representation of the source sentence of the source sample includes: performing importance sampling to change the weight of the source sample into an importance weight.

[0126] In Example 6, Example 5 may further include, wherein the importance weight is determined based on: a marginal distribution probability of the sample being classified as being from the source corpus, and a marginal distribution probability of the sample being classified as being from the target corpus.

[0127] In Example 7, at least one of Examples 1 to 6 may further include, wherein the machine learning model includes a neural network (NN) autoencoder, and wherein the operation further includes: modifying the representation of a source sentence of a source sample from a source corpus to include a representation of a target sentence of a target sample from a target corpus, including adversarially training the NN to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss.

[0128] In Example 8, Example 7 may also include, wherein the NN autoencoder includes a sample representation NN, which sample representation NN: implements a sequence-to-sequence encoder, which sequence-to-sequence encoder reads the embedded representation of the target sample in a first direction; creates a first hidden state sequence; reads the embedded representation of the target sample in a second direction opposite to the first direction; and creates a second hidden state sequence.

[0129] In Example 9, Example 8 may further include, wherein the sample representation NN provides a concatenation of a last hidden state of the embedded representation read in the first direction and a first hidden state of the embedded representation read in the second direction.

[0130] In Example 10, at least one of Examples 1 to 9 may further include, wherein the task classification further includes: training a machine learning model using the modified representation of the source sentence.

[0131] In Example 11, Example 10 may further include, wherein the task classification further includes: receiving feedback data from the PIM, and further training the model based on the feedback data, wherein the feedback data indicates whether the target sample includes the task based on user input of an interface of the PIM.

[0132] In Example 12, at least one of Examples 1 to 11 may further include, wherein the reminder comprises (i) an appointment, (ii) a meeting, (iii) a notification, (iv) a text message, (v) an email, or (vi) an instant message.

[0133] Example 13 may include at least one machine-readable storage medium comprising instructions for execution by a processing circuit system to perform operations for task detection, the operations comprising: modifying a representation of a source sentence of a source sample from a source corpus to more closely resemble a representation of a target sentence of a target sample from a target corpus, using a machine learning model to operate with the target sample to generate a task tag, the machine learning model being trained using the modified representation of the source sentence, the task tag indicating whether the target sample includes the task, and causing a personal information manager (PIM) to generate a reminder based on whether the target sample includes the task.

[0134] In Example 14, Example 13 may further include, wherein modifying the representation of the source sentence of the source sample includes: performing feature adaptation to change the features of the source sample to one or more specified values, performing importance sampling to change the weights of the source sample to importance weights, or using an adversarially trained neural network (NN) autoencoder to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss.

[0135] In Example 15, Example 14 may also include, wherein feature adaptation includes: training a corpus classifier machine learning model to distinguish between source samples and target samples; selecting top K features from the corpus classifier machine learning model; and replacing each of the top K features in the target sample with a specified value.

[0136] In Example 16, at least one of Examples 14 to 15 also includes, wherein modifying the representation of the source sentence of the source sample includes: identifying a mapping between features of the source sample and features of the target sample based on a distance between words of the source sample and corresponding mapping vectors; and replacing the representation of the words of the source sample with a mapping vector in the corresponding mapping vector.

[0137] In Example 17, at least one of Examples 14 to 16 also includes, wherein the importance weight is determined based on: a marginal distribution probability of the sample being classified as being from the source corpus, and a marginal distribution probability of the sample being classified as being from the target corpus.

[0138] In Example 18, at least one of Examples 13 to 17 may also include, wherein the machine learning model includes a neural network (NN) autoencoder, and wherein the operation also includes: modifying the representation of a source sentence of a source sample from a source corpus to include a representation of a target sentence of a target sample from a target corpus, including adversarially training the NN to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss.

[0139] In Example 19, Example 18 may also include, wherein the NN autoencoder includes a sample representation NN, which sample representation NN: implements a sequence-to-sequence encoder, which reads the embedded representation of the target sample in a first direction, creates a first hidden state sequence, reads the embedded representation of the target sample in a second direction opposite to the first direction, and creates a second hidden state sequence.

[0140] In Example 20, Example 19 may further include, wherein the sample representation NN provides a concatenation of a last hidden state of the embedded representation read in the first direction and a first hidden state of the embedded representation read in the second direction.

[0141] In Example 21, at least one of Examples 13 to 20 may further include, wherein the task classification further includes: training a machine learning model using the modified representation of the source sentence.

[0142] In Example 22, Example 21 may further include, wherein the task classification further includes: receiving feedback data from the PIM, and further training the model based on the feedback data, wherein the feedback data indicates whether the target sample includes the task based on user input of an interface of the PIM.

[0143] In Example 23, at least one of Examples 13 to 22 may further include, wherein the reminder comprises (i) an appointment, (ii) a meeting, (iii) a notification, (iv) a text message, (v) an email, or (vi) an instant message.

[0144] Example 24 includes a method performed by at least one processor of a computing system, the method comprising: modifying a representation of a source sentence of a source sample from a source corpus to more closely resemble a representation of a target sentence of a target sample from a target corpus, using a machine learning model to operate with the target sample to generate a task tag, the machine learning model being trained using the modified representation of the source sentence, the task tag indicating whether the target sample includes a task, and causing a personal information manager (PIM) to generate a reminder based on whether the target sample includes the task.

[0145] In Example 25, Example 24 may further include, wherein modifying the representation of the source sentence of the source sample includes: performing feature adaptation to change the features of the source sample to one or more specified values, performing importance sampling to change the weights of the source sample to importance weights, or using an adversarially trained neural network (NN) autoencoder to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss.

[0146] In Example 26, Example 25 may further include, wherein feature adaptation includes: training a corpus classifier machine learning model to distinguish between source samples and target samples; selecting top K features from the corpus classifier machine learning model; and replacing each of the top K features in the target sample with a specified value.

[0147] In Example 27, at least one of Examples 25 to 26 also includes, wherein modifying the representation of the source sentence of the source sample includes: identifying a mapping between features of the source sample and features of the target sample based on a distance between words of the source sample and corresponding mapping vectors; and replacing the representation of the words of the source sample with a mapping vector in the corresponding mapping vector.

[0148] In Example 28, at least one of Examples 25 to 27 also includes, wherein the importance weight is determined based on: a marginal distribution probability of the sample being classified as being from the source corpus, and a marginal distribution probability of the sample being classified as being from the target corpus.

[0149] In Example 29, at least one of Examples 24 to 28 may also include, wherein the machine learning model includes a neural network (NN) autoencoder, and wherein the operation also includes: modifying the representation of a source sentence of a source sample from a source corpus to include a representation of a target sentence of a target sample from a target corpus, including adversarially training the NN to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss.

[0150] In Example 30, Example 29 may also include, wherein the NN autoencoder includes a sample representation NN, which sample representation NN: implements a sequence-to-sequence encoder, which sequence-to-sequence encoder reads the embedded representation of the target sample in a first direction; creates a first hidden state sequence; reads the embedded representation of the target sample in a second direction opposite to the first direction; and creates a second hidden state sequence.

[0151] In Example 31, Example 30 may further include, wherein the sample representation NN provides a concatenation of a last hidden state of the embedded representation read in the first direction and a first hidden state of the embedded representation read in the second direction.

[0152] In Example 32, at least one of Examples 24 to 31 may further include, wherein the task classification further includes: training the machine learning model using the modified representation of the source sentence.

[0153] In Example 33, Example 32 may further include, wherein the task classification further includes: receiving feedback data from the PIM, and further training the model based on the feedback data, wherein the feedback data indicates whether the target sample includes the task based on user input of an interface of the PIM.

[0154] In Example 34, at least one of Examples 24 to 33 may further include, wherein the reminder comprises (i) an appointment, (ii) a meeting, (iii) a notification, (iv) a text message, (v) an email, or (vi) an instant message.

[0155] Although some embodiments have been described in detail above, other modifications are possible. For example, the logic flows depicted in the accompanying drawings do not require the order shown or the sequential order to achieve the desired results. Other steps can be provided or steps can be excluded from the described processes, and other components can be added to the described systems or removed from the described systems. Other embodiments may be within the scope of the following claims.

Claims

1. A system for task classification, comprising: Memory; A processing circuit system is coupled to the memory to implement an operation for performing task classification on a target sample from a target corpus of target samples, wherein the target corpus of the target sample includes a first dictionary and a context, wherein the task classification includes: modifying a representation of a source sentence of a source sample from a source corpus of a source sample according to a target sample characteristic to more closely resemble a target sentence of a target sample from a target corpus, the source corpus of the source sample comprising a second lexicon and context different from the first lexicon and context, wherein the target corpus of the target sample and the source corpus of the source sample are from different domains; using a machine learning model trained using the modified representation of the source sentence to operate with the target sample input to the machine learning model to generate a task label, the task label indicating whether the target sample includes a task; and The personal information manager PIM is enabled to generate a notification reminder based on the task in the target sample.

2. The system of claim 1, wherein modifying the representation of the source sentence of the source sample comprises: Feature adaptation is performed to change the features of the source samples to one or more specified values.

3. The system of claim 2, wherein the feature adaptation comprises: Training a corpus classifier machine learning model to distinguish between the source samples and the target samples; Selecting the top K features from the corpus classifier machine learning model; as well as Each of the first K features in the target sample is replaced with the specified value.

4. The system of claim 2, wherein modifying the representation of the source sentence of the source sample comprises: identifying a mapping between a feature of the source sample and a feature of the target sample based on a distance between a word of the source sample and a corresponding mapped target word of the target sample; as well as The vector representation of the corresponding word of the source sample is replaced by the mapping vector of the corresponding mapped target word.

5. The system of claim 1 , wherein modifying the representation of the source sentence of the source sample comprises: Importance sampling is performed to change the weights of the source samples into importance weights.

6. The system of claim 5, wherein the importance weight is determined based on: a marginal distribution probability of the sample being classified as being from the source corpus and a marginal distribution probability of the sample being classified as being from the target corpus.

7. The system of claim 1, wherein the machine learning model comprises a neural network (NN) autoencoder, and wherein the operations further comprise: Modifying a representation of a source sentence of a source sample from a source corpus to include a representation of a target sentence of the target sample from the target corpus includes adversarially training the NN to increase the corpus classification loss while reducing the task classification loss and the sample reconstruction loss.

8. A system according to claim 7, wherein the NN autoencoder includes a sample representation NN, which implements a sequence-to-sequence encoder, which reads the embedded representation of the target sample in a first direction; creates a first hidden state sequence; reads the embedded representation of the target sample in a second direction opposite to the first direction; and creates a second hidden state sequence.

9. The system of claim 8, wherein the sample representation NN provides a concatenation of a last hidden state of the embedded representation read in the first direction and a first hidden state of the embedded representation read in the second direction.

10. The system according to claim 1, wherein the task classification further comprises: The machine learning model is trained using the modified representation of the source sentence.

11. The system according to claim 10, wherein the task classification further comprises: Feedback data is received from the PIM, and the model is also trained based on the feedback data, wherein the feedback data indicates whether the target sample includes the task based on user input of an interface of the PIM.

12. The system of claim 1, wherein the reminder comprises (i) an appointment, (ii) a meeting, (iii) a text message, (iv) an email, or (v) an instant message.

13. At least one machine-readable storage medium, comprising instructions for execution by a processing circuit system to perform operations for task detection in a target sample from a target corpus of target samples, the target corpus of target samples comprising a first lexicon and a context, the operations comprising: modifying a representation of a source sentence of a source sample from a source corpus of a source sample according to a target sample characteristic to more closely resemble a representation of a target sentence of a target sample from a target corpus, the source corpus of the source sample comprising a second lexicon and context different from the first lexicon and context, wherein the target corpus of the target sample and the source corpus of the source sample are from different domains; using a machine learning model trained using the modified representation of the source sentence to operate with the target sample input to the machine learning model to generate a task label, the task label indicating whether the target sample includes a task; and The personal information manager PIM is enabled to generate a notification reminder based on the task in the target sample.

14. The at least one machine-readable storage medium of claim 13, wherein modifying the representation of the source sentence of the source sample comprises: Perform feature adaptation to change the features of the source samples to one or more specified values, perform importance sampling to change the weights of the source samples to importance weights, or use an adversarially trained neural network (NN) autoencoder to increase corpus classification loss while reducing task classification loss and sample reconstruction loss.

15. The at least one machine-readable storage medium of claim 14, wherein the feature adaptation comprises: Training a corpus classifier machine learning model to distinguish between the source samples and the target samples; Selecting the top K features from the corpus classifier machine learning model; as well as Each of the first K features in the target sample is replaced with the specified value.

16. The at least one machine-readable storage medium of claim 14, wherein modifying the representation of the source sentence of the source sample comprises: Identifying a mapping between a feature of the source sample and a feature of the target sample based on a distance between a word of the source sample and a corresponding mapping vector; as well as The representation of the word of the source sample is replaced by a mapping vector of the corresponding mapping vector.

17. The at least one machine-readable storage medium of claim 14, wherein the importance weight is determined based on: a marginal distribution probability of a sample being classified as being from the source corpus and a marginal distribution probability of the sample being classified as being from the target corpus.

18. A method executed by at least one processor of a computing system for performing task classification in a target sample from a target corpus of target samples, the target corpus of the target sample comprising a first dictionary and a context, the method comprising: modifying a representation of a source sentence of a source sample from a source corpus of a source sample according to a target sample characteristic to more closely resemble a representation of a target sentence of a target sample from a target corpus, the source corpus of the source sample comprising a second lexicon and context different from the first lexicon and context, wherein the target corpus of the target sample and the source corpus of the source sample are from different domains; using a machine learning model trained using the modified representation of the source sentence to operate with the target sample input to the machine learning model to generate a task label, the task label indicating whether the target sample includes a task; and The personal information manager PIM is enabled to generate a notification reminder based on the task in the target sample.

19. The method according to claim 18, wherein the task classification further comprises: The machine learning model is trained using the modified representation of the source sentence.

20. The method according to claim 19, further comprising: Feedback data is received from the PIM, and the model is also trained based on the feedback data, wherein the feedback data indicates whether the target sample includes the task based on user input of an interface of the PIM.

Images