General opinion summaries for medical decision-making

Abstract

A method and system for document summarization includes dividing a document into sentences (602) and sorting the sentences by a metric that promotes the spread of review opinions from the document to generate a ranked list of sentences. Groups of sentences with similar embeddings are formed (614), and a trained generalization encoder-decoder model is applied (616) to output common generalizations for the sentences in each group. Sentences are added to the summary in rank order (608) from generalizations corresponding to sentences in the ranked list until a target summary length is reached. Actions are performed (612) in response to the summaries.

Description

[Technical Field]

[0001] Related Application Information This application claims priority to U.S. Patent Application No. 63 / 484,534, filed February 13, 2023, U.S. Patent Application No. 63 / 496,446, filed April 17, 2023, U.S. Patent Application No. 63 / 532,340, filed August 11, 2023, U.S. Patent Application No. 63 / 533,399, filed August 18, 2023, and U.S. Patent Application No. 18 / 439,274, filed February 12, 2024, each of which is incorporated by reference in its entirety. [Background technology]

[0002] The present invention relates to natural language processing, and more particularly to machine learning models for summarizing opinions. 2. Description of Related Art

[0003] Opinion summarization is a natural language processing task that attempts to identify the most prominent opinions expressed in a collection of documents. However, existing approaches to summarization do not distinguish between common opinions and opinions that are more rarely expressed. Summary of the Invention

[0004] A method for document summarization includes dividing a document into sentences and sorting the sentences by a metric that promotes the spread of review opinions from the document to generate a ranked list of sentences. Groups of sentences with similar embeddings are formed, and a trained generalized encoder-decoder model is applied to output a common generalization of the sentences in each group. Sentences are added to the summary in rank order from the generalizations that correspond to the sentences in the ranked list until a target summary length is reached. Actions corresponding to the summary are performed.

[0005] A system for document generalization includes a hardware processor and a memory that stores a computer program. When executed by the hardware processor, the computer program causes the hardware processor to partition a document into sentences, sort the sentences by a metric that drives the dissemination of review opinions from the document, generate a ranked list of the sentences, form groups of sentences with similar embeddings, apply a trained generalization encoder-decoding model to output common generalizations for the sentences in each group, add sentences to a summary in rank order from the generalizations that correspond to the sentences in the ranked list until a target summary length is reached, and perform actions responsive to the summaries.

[0006] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0007] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0008] [Figure 1] FIG. 2 is a block diagram of training and using a document summarization model according to one embodiment of the present invention.

[0009] [Figure 2] 1 is pseudocode for an exemplary greedy summarization process according to one embodiment of the present invention.

[0010] [Figure 3] FIG. 2 is a block diagram of an exemplary opinion summarization model, according to one embodiment of the present invention.

[0011] [Figure 4] FIG. 2 is a block diagram of an exemplary opinion generalization model, according to one embodiment of the present invention.

[0012] [Figure 5]1 is a block / flow diagram of a method for summarizing opinions, according to one embodiment of the present invention;

[0013] [Figure 6] 1 is a block / flow diagram of a method for summarizing opinions, according to one embodiment of the present invention;

[0014] [Figure 7] FIG. 2 is a block / flow diagram of a method for training an opinion summarization model according to one embodiment of the present invention.

[0015] [Figure 8] FIG. 1 is a block diagram illustrating a review summary in the context of a healthcare facility, according to one embodiment of the present invention.

[0016] [Figure 9] FIG. 2 is a block diagram of a computing device capable of performing review summarization, according to one embodiment of the present invention.

[0017] [Figure 10] FIG. 2 is a diagram of an exemplary neural network architecture that can be used to implement a portion of a generalization model, according to one embodiment of the present invention.

[0018] [Figure 11] FIG. 2 is a diagram of an exemplary neural network architecture that can be used to implement a portion of a generalization model, according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] To derive opinion summaries from a collection of documents, an automatic, reference-free metric for measuring opinion prevalence may be used. This metric may give greater weight to opinions expressed more frequently in the documents. Furthermore, opinions may be sorted according to how informative they are, based on measures of triviality and redundancy. We describe a summary method that emphasizes common opinions. When used in conjunction with other metrics such as fluency and coherence, it can generate summaries that better capture reliable and informative opinions from diverse collections.

[0020] Opinion summarization extracts salient opinions from a collection of reviews of a product, service, or other object. However, obtaining a large training set of summarizations is challenging, especially one that provides a diverse set of reliable opinions about various products. For large data sets, it may be impossible to provide human-generated references as a training guide, since humans cannot memorize all of the source text at once.

[0021] Referring now to FIG. 1, a high-level diagram of the training and operation of an opinion summarization model is shown. A training corpus 102 consisting of a set of training documents 103 is used in training 104 to generate a trained model 106. During operation, a test corpus 108 consisting of documents 109 is input to the trained model 106. The trained model 106 generates a summary 110 of the test corpus. The summary 110 may consist of a list of sentences that reflect the sentiment expressed in the test corpus 108. Additionally, an importance ranking 112 may be generated that ranks the sentences in the summary 110 by their importance.

[0022] Various metrics can be used to determine the quality of a given summary. In practice, multiple such metrics can be used to determine an overall quality score. In this embodiment, a metric incorporating opinion prevalence can be used, which weights how many times a given opinion appears in the test corpus 108.

[0023] Given a binary classifier C(x,y) that returns 1 if text x logically entails text y and 0 otherwise, we can classify a set of reviews R={x1,...,x m} sentences about y1,...,y n For a summary S with , we can define an opinion prevalence score. The opinion prevalence should reward opinions expressed by multiple source reviews. For a given text y, this can be based on quantity.

number

[0024] For summaries longer than one sentence, there are two masks that are applied to the classifier values: one mask prevents statements that have already been mentioned from being counted, to avoid redundancy. k , then this mask is expressed as:

number

[0025] The second mask may block trivial conclusions that can be drawn from the fact that someone purchased the indicated product or service without providing any additional information about that person's experience. For example, sentence t is "I buy p." If p is a sneaker, obvious conclusions such as "It's a shoe" or "I wear it" can be logically expressed by all reviews without providing any useful information. Thus, C(x i ,y) may be masked by 1-C(t,y). If product or service names are not available, this mask may be omitted.

[0026] Applying these masks to the scoring formulation, the definition of opinion prevalence can be expressed as follows:

number

number

number

[0027] Opinion prevalence may be compared between summaries of similar length, or shorter summaries may be favored if the most common opinions can be selected. Opinion prevalence provides a scoring of opinion summary output without the need for a reference summary.

[0028] Referring now to Figure 2, pseudocode for a summarization strategy that maximizes opinion prevalence in the output summary is shown. This method produces summaries with high opinion prevalence as measured by the methods described above, and may actually produce summaries with higher prevalence scores than human-generated summaries.

[0029] Most opinion summarization systems rely on dividing input reviews into sentences and considering the expressions and characteristics of each sentence. In reality, input sentences can be long and complex, combining multiple different opinions. This complexity can make it difficult to extract and associate common arguments from various reviews. Text simplification can be used to preprocess the input sentences of source reviews.

[0030] Referring now to Figure 3, there is shown a diagram of an opinion comparison model 300. A multi-layer perceptron M302 is connected to an encoder E P 304 (for premise p) and E H 306 (for hypothesis h) and comparison features from the encoder E. P and E H may have the same weight. M is trained for a binary classification task of entailment or non-entailment. The term "entailment" may be understood to be equivalent to the logical entailment determined in the NLI task. The comparison features may be expressed as conjunctions (x; y; |xy|; x*y), where x = E P (p), y=E H(h), * is the element-wise product. M is implemented as a two-layer perceptron with 128 hidden units and rectified linear unit (ReLU) nonlinearity, and can output two logits. The encoder can be a transencoder model such as the Electra-based model. The encoder is trained end-to-end with M on a mixture of binarized but not symmetrized NLI datasets.

[0031] For opinion summarization, we can collect groups of sentences with similar meanings from reviews of a single product or service. P (x)=E H (x) is pre-computed for every sentence x contained in a product review. The similarity measure cos(E(x),E(y)) is determined for every pair of sentences (x,y) in the input review.

[0032] The number of comparisons grows quadratically with the number of sentences, but the transformation network only needs to be run once for each sentence. Alternatively, the computation can be performed as M(E(x);E(y);|E(x)-E(y)|;E(x)*E(y)), where it is not symmetric with respect to x and y. M can be implemented as a small neural network that is computationally less expensive than the transformer E.

[0033] Sentence ranking may be implemented as a weighted count for each sentence of the number of other sentences that entail or are entailed by it. This ranking does not allow for a threshold for entailment similarity. Thus, if a product review is found in sentences x1...x2, n , the ranking score is

number

number

number

number

[0034] Although E is trained on M and not on cos for asymmetric NLI instances, both cos and M have similar accuracy at the best threshold in predicting symmetrized relations between sentences. For example, if either x entails y or y entails x, a positive symmetry relation is found.

[0035] sentence

number

[0036] Instead of taking an extractive summary from the embeddings, for each k=1,...,m, we extract a sentence x j Group of

number

number

[0037] Training data for this sequence-to-sequence model is established by first running the NLI model on each review sentence pair, e.g., up to eight reviews of the same product. The training set may be divided into a training subset (e.g., 80%), a development subset (e.g., 10%), and a test subset (e.g., 10%). Weakly implied decisions may be understood as those that yield probabilities below a first threshold, and strongly implied decisions may be understood as those that yield probabilities above a second threshold.

[0038] Given these entailment decisions, sentences may be sorted by the number of other sentences that strongly entail them. Sentences may be selected in order such that they are not weakly entailed by previously selected sentences, as long as each is strongly entailed by at least one other sentence.

[0039] Each selected example becomes the target sequence in the example, and the source sequence is a set of sentences that strongly implicate it, connected by vertical bars. A model may be trained using cross-entropy loss to predict the target sequence from the source sequence.

[0040] 4, there is shown a generalization model 400. The trained generalization model 400 is

number

[0041] Sentence y1,...,y n The informativeness of a summary S with m reviews R={x1,...,x m}, m' reviews of other products p'≠p in the same category as p, R'={x'1,...,x' m’ In contrast to the set of}, it may be defined as follows:

number

[0042] Referring now to Figure 5, a method for summarizing opinions from a set of reviews is shown. Block 502 divides each document corresponding to a respective review of a particular product or service into sentences, which are optionally simplified by block 504, as described above, to separate complex sentiments into more simplified expressions. Block 506 filters out trivial conclusions that logically follow from the fact that someone purchased the product or service in question.

[0043] Block 508 counts the number of other documents that implicate each document according to the NLI model and sorts the sentences according to the number of implications. Block 510 then selects the highest-ranked remaining sentence from the sorted sentences. Starting with an empty summary, block 512 determines whether the selected sentence is implicated by anything already in the summary. If not, block 514 adds the sentence to the summary. Block 514 determines whether the summary has reached a target length. If not, processing returns to block 510, where the next most implicated sentence is selected. If so, processing ends, and block 518 outputs the sentences added to the summary.

[0044] Referring now to Figure 6, a method for generating summaries through embedding is shown. Block 602 segments a review document into sentences, as described above. Block 604 embeds each sentence using encoder E, rendering the sentences as their respective vector representations. Block 606 calculates the discount score for each sentence, as described above, and sorts the sentences accordingly. Block 608 adds the sentence with the highest score to the summary, and block 610 checks whether the summary has reached a target length. If not, block 608 adds the next highest scoring sentence to the summary. This process is repeated until the target length is reached, at which point block 612 outputs the selected sentence as the summary.

[0045] Further generalization may optionally be performed on the summary. Block 614 identifies the groups (e.g.,

number

[0046] Referring now to FIG. 7, a method for training a generalization model is shown. A collection of reviews potentially related to multiple products or services is used as training data. For a first product or service, block 702 uses the trained NLI model to determine entailments between pairs of review sentences. Block 704 sorts the sentences by the number of strong entailments (e.g., probabilities output by the NLI model exceed a strong entailment threshold). Block 706 then selects sentences that are not weakly entailed by previous selections (e.g., have probabilities output by the NLI model below a weak entailment threshold).

[0047] For each selected sentence, block 708 collects sentences that strongly implicate it as a source sequence. Block 710 then trains a generalization model using cross-entropy loss to predict targets from source sequences.

[0048] Block 712 determines whether there are additional products or services to use for training. If so, block 716 selects the next product or service, and block 702 determines the implications between review sentences related to the next product. If all products / services have been used for training, block 714 outputs the trained generalization model.

[0049] 8, an illustration of a review summary is shown in the context of a healthcare facility 800. The review summary 808 may be used to guide the decision-making of a healthcare professional 802, for example, in determining which healthcare products are appropriate for treating a given patient or for use in the healthcare facility 800. The automated review summary helps the healthcare professional 802 understand the costs and benefits of a given product in the context of the patient's medical condition.

[0050] The healthcare facility may include one or more healthcare professionals 802 that review information from the patient's medical record 806 to determine the patient's healthcare and treatment needs. The treatment system 804 may further be designed to monitor the patient's condition to generate the medical record 806 and automatically manage and adjust treatment as needed. In a particular example, the review may be regarding supplements or medications that may be provided to the user, such as vitamins or nutritional supplements. In another example, the review may relate to mobility aids, such as canes, walkers, wheelchairs, or prosthetic limbs.

[0051] The review summary 808 may identify a set of important general opinions about the product based on information drawn from reviews of one or more related products. The medical professional 802 may then make decisions about the patient's health care based on the review summary, such as determining which product is most effective for the patient's particular needs.

[0052] The different elements of the healthcare facility 800 can communicate with each other over a network 810, for example, using any suitable wired or wireless communication protocol and medium. Thus, the review summary 808 transmits a report to a healthcare professional 802, who may make healthcare decisions in the context of the patient's medical record 806. In some cases, the review summary 808 may be integrated with an automated treatment system 804 to automatically trigger changes in the patient's treatment in response to information obtained from the review summary. For example, if the review summary indicates that a certain product is unsafe, the treatment system 804 may automatically suspend treatment with that product until it can be reviewed by a healthcare professional 802.

[0053] 9, an exemplary computing device 900 is shown, in accordance with an embodiment of the present invention. The computing device 900 is configured to perform review summarization.

[0054] Computing device 900 may be embodied as any type of computational or computing device capable of performing the functions described herein, such as, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and / or a user electronic device. Additionally or alternatively, computing device 900 may be embodied as one or more compute sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically separated computing device.

[0055] 9, computing device 900 illustratively includes a processor 910, an input / output subsystem 920, a memory 930, a data storage device 940, and a communications subsystem 950, and / or other components and devices typically found in a server or similar computing device. In other embodiments, computing device 900 may include other or additional components (e.g., various input / output devices) such as those typically found in a server computer. Furthermore, in some embodiments, one or more of the illustrative components may be incorporated into or otherwise form part of another component. For example, memory 930, or portions thereof, may be incorporated into processor 910 in some embodiments.

[0056] The processor 910 may be embodied as any type of processor capable of performing the functions described herein, including a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0057] Memory 930 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, memory 930 may store various data and software used during operation of computing device 900, such as an operating system, applications, programs, libraries, and drivers. Memory 930 is communicatively coupled to processor 910 via I / O subsystem 920 and may be embodied as circuits and / or components for facilitating input / output operations with processor 910, memory 930, and other components of computing device 900. For example, I / O subsystem 920 may be embodied as or otherwise include a memory controller hub, an input / output control hub, a platform controller hub, integrated control circuitry, firmware devices, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems for facilitating input / output operations. In some embodiments, I / O subsystem 920 may form part of a system-on-chip (SOC) and be integrated with processor 910, memory 930, and other components of computing device 900 on a single integrated circuit chip.

[0058] The data storage device 940 may be embodied as any type of device or device configured for short-term or long-term storage of data, such as, for example, a memory device and circuitry, a memory card, a hard disk drive, a solid-state drive, or other data storage device. The data storage device 940 may store program code 940A for model training, program code 940B for performing review summaries, and / or program code 940C for performing automated actions responsive to the review summaries. The communications subsystem 950 of the computing device 900 may be embodied as any network interface controller or other communications circuitry, device, or collection thereof that can enable communications between the computing device 900 and other remote devices over a network. The communications subsystem 950 may be configured to achieve such communications using any one or more communications technologies (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, InfiniBand, Bluetooth, Wi-Fi, WiMAX, etc.).

[0059] As shown, the computing device 900 may also include one or more peripheral devices 960. The peripheral devices 960 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, the peripheral devices 960 may include a display, a touch screen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0060] Of course, computing device 900 may include other elements (not shown) or omit certain elements, as would readily occur to one skilled in the art. For example, various other sensors, input devices, and / or output devices may be included in computing device 900, depending on the particular implementation of the same, as would be readily understood by one skilled in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Furthermore, additional processors, controllers, memory, etc. may be utilized in a variety of configurations. These and other variations of processing system 900 will be readily contemplated by one skilled in the art in view of the teachings of the present invention provided herein.

[0061] 10 and 11, exemplary neural network architectures are shown, which may be used to implement portions of the present model, such as MLP 302. Neural networks are generalized systems whose functionality and accuracy improve with exposure to additional empirical data. Neural networks learn by exposure to empirical data. During training, the neural network stores and adjusts weights that are applied to input empirical data. Applying the adjusted weights to the data can identify that data belongs to a specific predefined class from a set of classes and output the probability that the input data belongs to each class.

[0062] Empirical data (also called training data) from a set of examples is formatted as a string of values ​​and fed to the input of a neural network. Each example is associated with a known outcome or output. Each column is represented as an (x,y) pair, where x represents the input data and y represents the known output. The input data can be of various data types and contain multiple different values. The network can have one input node for each value that makes up the example input data, and a separate weight can be applied to each input value. The input data can be formatted, for example, as a vector, array, or string, depending on the architecture of the neural network being built and trained.

[0063] Neural networks "learn" by comparing neural network outputs generated from input data with known values ​​from examples and adjusting stored weights to minimize the difference between the output value and the known value. Adjustments can be made to the stored weights through backpropagation, where the weight's influence on the output value is determined by calculating a mathematical gradient and adjusting the weight in a way that shifts the output toward the minimum difference. This optimization, called gradient descent, is a non-limiting example of how training occurs. A subset of examples with known values ​​that were not used in training can be used to test and validate the accuracy of the neural network.

[0064] During operation, a trained neural network can be used on new data not previously used for training or validation through generalization. The adjusted neural network weights can be applied to the new data, and the weights estimate the function developed from the training examples. The parameters of the estimated function, captured by the weights, are based on statistical inference.

[0065] In a layered neural network, nodes are arranged in layers. An exemplary simple neural network has an input layer 1020 of source nodes 1022 and a single computational layer 1030 with one or more computational nodes 1032 that also function as output nodes, with a single computational node 1032 for each possible category into which an input example may be classified. The input layer 1020 may have a number of source nodes 1022 equal to the number of data values ​​1012 in the input data 1010. The data values ​​1012 in the input data 1010 may be represented as a column vector. Each computational node 1032 in the computational layer 1030 generates a weighted linear combination of values ​​from the input data 1010 provided to the input node 1020 and applies a differentiable nonlinear activation function to the sum. An exemplary simple neural network can perform classification on linearly separable examples (e.g., patterns).

[0066] A deep neural network, such as a multilayer perceptron, may have an input layer 1020 of source nodes 1022, one or more computational layers 1030 with one or more computational nodes 1032, and an output layer 1040 with one output node 1042 for each possible category into which an input example may fall. The input layer 1020 may have a number of source nodes 1022 equal to the number of data values ​​1012 in the input data 1010. The computational nodes 1032 in the computational layer 1030 are also referred to as hidden layers because they are located between the source nodes 1022 and the output nodes 1042 and are not directly observed. Each node 1032, 1042 in the computational layer generates a weighted linear combination of values ​​from the values ​​output by the nodes in the previous layer and applies a nonlinear activation function that is differentiable over the range of the linear combination. The weights applied to the values ​​from each previous node may be, for example, w1, w2, ... w n-1 ,w nThe output layer provides the network's overall response to the input data. Deep neural networks can be fully connected, where each node in a computational layer is connected to every node in the previous layer, or the connections between layers can have other configurations. If links between nodes are missing, the network is said to be partially connected.

[0067] Training a deep neural network involves two phases: a forward phase, in which the weights of each node are fixed and inputs are propagated through the network, and a backward phase, in which error values ​​are propagated back through the network and the weight values ​​are updated.

[0068] Computational nodes 1032 in one or more computational (hidden) layers 1030 perform nonlinear transformations on the input data 1012 to generate a feature space. Classes or categories may be more easily separated in the feature space than in the original data space.

[0069] The embodiments described herein may be entirely hardware, entirely software, or contain both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0070] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. A computer-usable or computer-readable medium may include any apparatus that stores, communicates, propagates, or transports a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or propagation medium. The medium may include computer-readable storage media such as semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks.

[0071] Each computer program can be tangibly stored on a machine-readable storage medium or device (e.g., program memory or magnetic disk) readable by a general-purpose or special-purpose programmable computer to configure and control the operation of the computer when the storage medium or device is read by the computer to perform the procedures described herein. The system of the present invention can also be considered to be embodied in a computer-readable storage medium configured with a computer program, where the configured storage medium causes the computer to operate in a particular, predetermined manner to perform the functions described herein.

[0072] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements via a system bus. The memory elements may include local memory employed during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some program code to reduce the number of times the code is retrieved from bulk storage during execution. Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I / O controllers.

[0073] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0074] As employed herein, the terms “hardware processor subsystem” or “hardware processor” can refer to a processor, memory, software, or combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, a hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The data processing elements may be included in a central processing unit, a graphics processing unit, and / or a separate processor- or computing element-based controller (e.g., logic gates, etc.). A hardware processor subsystem may include one or more on-board memories (e.g., cache, dedicated memory array, read-only memory, etc.). In some embodiments, a hardware processor subsystem may include one or more memories (e.g., ROM, RAM, basic input / output system (BIOS), etc.) that may be on-board or off-board or dedicated for use by the hardware processor subsystem.

[0075] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code for achieving a specified result.

[0076] In other embodiments, the hardware processor subsystem may include dedicated, specialized circuitry that performs one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0077] These and other variations of the hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0078] References herein to "one embodiment" or "one embodiment" of the present invention, as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in one embodiment," as well as any other variations thereof, in various places throughout this specification do not necessarily all refer to the same embodiment. However, it should be understood that features of one or more embodiments may be combined given the teachings of the present invention provided herein.

[0079] For example, in the case of "A / B," the use of any of the following " / ," "and / or," "at least one," such as "A and / or B" or "at least one of A and B" will be understood to be intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), the selection of only the first and third listed alternatives (A and C), the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as there are listed items.

[0080] The foregoing is understood in all respects to be illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description, but from the claims which are interpreted in accordance with the full breadth permitted by the patent laws. It will be understood that the embodiments shown and described herein are merely exemplary of the invention, and that those skilled in the art could make various modifications without departing from the scope and spirit of the invention. Various other feature combinations could be implemented by those skilled in the art without departing from the scope and spirit of the invention. Having thus described aspects of the invention with the detail and particularity required by the patent laws, what is desired to be claimed and protected by Letters Patent is set forth in the appended claims.

Claims

1. 1. A computer-implemented method for document summarization, comprising: Segmenting the document into sentences (602); sorting (606) the sentences by a metric that promotes the dissemination of review opinions from the document to generate a ranked list of sentences; forming 614 groups of sentences with similar embeddings and applying 616 the trained generalized encoder-decoder model to output a common generalization of the sentences in each group; adding (608) sentences from the generalizations corresponding to the sentences in the ranked list to the summary in rank order until a target summary length is reached; and performing (612) an action responsive to the summary.

2. 10. The method of claim 1, The method further comprising simplifying the sentences before sorting.

3. 10. The method of claim 1, The method further comprising filtering trivial conclusions from said sentences before sorting.

4. 10. The method of claim 1, The method wherein sorting the sentences includes sorting by the number of implications each sentence has from the other sentences.

5. 10. The method of claim 1, A method in which sorting the sentences includes assigning a score to each sentence based on a cosine comparison of the encoded sentence representations.

6. 6. The method of claim 5, The score is further discounted by the maximum of each sentence's previous scores.

7. 10. The method of claim 1, The method, wherein the document is a review of a healthcare product, and performing the action includes modifying a patient's treatment based on the summary.

8. 8. The method of claim 7, The method, wherein altering treatment for the patient includes automatically discontinuing treatment indicated by the summary as being unsafe.

9. 8. The method of claim 7, The method, wherein altering treatment for the patient comprises assisting a medical professional in making a decision.

10. 10. The method of claim 1, A method in which generalizing the summary includes ranking the summary's informativeness based on a comparison of its implications from reviews of a current product or entity and its implications from reviews of a different product or entity.

11. 1. A system for a document summarization system, comprising: a hardware processor (910); and a memory (9409) for storing a computer program, the computer program, when executed by the hardware processor, causing the hardware processor to: Segmenting the document into sentences (602); sorting (606) the sentences by a metric that promotes the dissemination of review opinions from the document to generate a ranked list of sentences; forming 614 groups of sentences with similar embeddings and applying 616 the trained generalized encoder-decoder model to output a common generalization of the sentences in each group; adding (608) sentences from the generalizations corresponding to the sentences in the ranked list to the summary in rank order until a target summary length is reached; performing (612) an action responsive to said summary; and a system for causing said action to be performed.

12. 12. The system according to claim 11, The computer program further causes the hardware processor to simplify the sentences before sorting.

13. 12. The system of claim 11, The computer program further causes the hardware processor to filter trivial conclusions from the sentences before sorting.

14. 12. The system of claim 11, The computer program further causes the hardware processor to sort the sentences by the number of implications each sentence has from other sentences.

15. 12. The system of claim 11, The computer program further causes the hardware processor to assign a score to each sentence based on a cosine comparison of the encoded sentence representations.

16. 16. The system of claim 15, The score is further discounted by the maximum of each sentence's previous scores.

17. 12. The system of claim 11, The document is a review of a healthcare product, and the computer program further causes the hardware processor to modify the patient's treatment based on the summary.

18. 18. The system of claim 17, The computer program further causes the hardware processor to automatically discontinue any treatment indicated by the summary as being unsafe.

19. 18. The system of claim 17, The computer program further causes the hardware processor to make a decision by a medical professional.

20. 12. The system of claim 11, The computer program further causes the hardware processor to rank the summary for informativeness based on a comparison of implications of the summary from reviews of a current product or entity and implications from reviews of a different product or entity.

Images