Generating natural language expression transformations

The system automatically generates natural language expression variants by using predetermined symbols and template structured representations to replace key entities, enhancing efficiency and accuracy in natural language processing tasks.

JP7595654B2Active Publication Date: 2024-12-06INTERNATIONAL BUSINESS MACHINE CORPORATION
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2022520904
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-10-31
Filing Date
2020-10-15
Publication Date
2024-12-06
Estimated Expiration
2040-10-15

Smart Images

  • Figure 0007595654000001
    Figure 0007595654000001
  • Figure 0007595654000002
    Figure 0007595654000002
  • Figure 0007595654000003
    Figure 0007595654000003
Patent Text Reader

Abstract

Embodiments of the present disclosure relate to generating natural language expression variants. In one embodiment, a computer-implemented method is disclosed. According to the method, a structured representation is determined for a source expression in natural language by replacing a source key entity in the source expression with a predetermined symbol. At least one template structured representation is selected from a set of template structured representations based on a respective similarity between the structured representation and each template structured representation in the set. Each of the set of template structured representations includes a predetermined symbol for representing a key entity. At least one variant representation is generated for the source expression by replacing a predetermined symbol in the at least one selected template structured representation with the source key entity. In another embodiment, a system and computer program product are disclosed.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to natural language processing techniques, and more particularly to generating natural language expression variants. [Background technology]

[0002] As technology develops, the use of natural language processing has expanded to encompass a wide variety of applications, including text generation, which can be used to enable human-computer dialogue, question answering, intent recognition, report generation, automated technical documentation, instruction customization, item and event descriptions, tutorials, and stories. Machine learning or deep learning models are widely used to improve natural language processing tasks. Summary of the Invention

[0003] According to one embodiment of the present invention, a computer-implemented method is provided. According to the method, a structured representation is determined for a source representation in a natural language by replacing source key entities in the source representation with predefined symbols. At least one template structured representation is selected from a set of template structured representations based on respective similarities between the structured representation and each template structured representation in the set. Each of the set of template structured representations includes predefined symbols for representing key entities. At least one variant representation is generated for the source representation by replacing predefined symbols in the at least one selected template structured representation with the source key entities.

[0004] According to a further embodiment of the present invention there is provided a system comprising a processing unit and a memory coupled to the processing unit and storing instructions which, when executed by the processing unit, perform operations of a method according to an embodiment of the present invention.

[0005] According to a further embodiment of the present invention, there is provided a computer program product tangibly stored on a non-transitory machine-readable medium and including machine-executable instructions that, when executed on a device, cause the device to perform operations of a method according to an embodiment of the present invention.

[0006] The above and other objects, features, and advantages of the present disclosure will become more apparent through a more detailed description of several embodiments of the present disclosure in the accompanying drawings. The same references generally refer to the same components in the embodiments of the present disclosure. [Brief description of the drawings]

[0007] [Figure 1] FIG. 2 illustrates a cloud computing node according to an embodiment of the present invention. [Diagram 2] FIG. 1 illustrates a cloud computing environment according to an embodiment of the present invention. [Diagram 3] FIG. 2 illustrates an abstract model layer according to an embodiment of the present invention. [Figure 4] 1 is a block diagram of an example system for generating natural language expression variants in accordance with an embodiment of the present invention. [Diagram 5] 5 is a block diagram of an example entity recognition module in the system of FIG. 4 in accordance with an embodiment of the present invention. [Figure 6] FIG. 2 is a block diagram of an example system for generating natural language expression variants in accordance with another embodiment of the present invention. [Figure 7] 7 is a block diagram of an example template addition module in the system of FIG. 6 in accordance with an embodiment of the present invention. [Figure 8] 1 is a flowchart of an example method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Some embodiments will be described in more detail with reference to the accompanying drawings, in which embodiments of the present disclosure are illustrated, however, the present disclosure may be embodied in various ways and therefore should not be construed as limited to the embodiments disclosed herein.

[0009] Although this disclosure includes a detailed description of cloud computing, it should be understood that implementations of the teachings recited herein are not limited to cloud computing environments. Rather, embodiments of the invention may be practiced in conjunction with any other type of computing environment now known or later developed.

[0010] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0011] The characteristics are as follows: On-Demand Self-Service: Cloud consumers can unilaterally provision computing capabilities such as server time and network storage automatically as needed, without the need for human interaction with the service provider. Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., cell phones, laptops, and PDAs). Resource Sharing: Provider computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated according to demand. Location independence is meaningful in that consumers generally have no control or knowledge over the exact location of the resources provided, although it may be possible to specify location at a higher level of abstraction (e.g., country, state, or data center). Rapid Scalability: Capabilities can be quickly and elastically provisioned, in some cases automatically, to instantly scale out, and quickly released to instantly scale in. To the consumer, the capabilities available for provisioning often appear unlimited and can be purchased in any quantity at any time. Services are meterable: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services being used.

[0012] The service model is as follows: Software as a Service (SaaS): The capability provided to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. Platform as a Service (PaaS): The capability provided to the consumer is to deploy consumer-created or consumer-acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does have control over the deployed applications and, to the extent possible, the application hosting environment configuration. Infrastructure as a Service (IaaS): The capability provided to the consumer is the supply of processing, storage, networking, and other basic computing resources onto which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating systems, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0013] The layout model is as follows: Private Cloud: The cloud infrastructure is operated exclusively for an organization. The cloud infrastructure may be managed by the organization or a third party and may be on-premise or off-premise. Community Cloud: The cloud infrastructure is shared by multiple organizations to support a specific community with shared concerns (e.g., mission, security requirements, policy, and compliance considerations). The cloud infrastructure may be managed by the organization or a third party and may reside on-premise or off-premise. Public Cloud: The cloud infrastructure is made available to the general public or large industry organizations and is owned by an organization that sells cloud services. Hybrid Cloud: The cloud infrastructure is a composite of two or more clouds (private, community, or public) that remain a unique entity but are joined by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0014] Cloud computing environments are stateless, low-coupling, modular, and service-oriented with a focus on semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0015] 1, a schematic of an example cloud computing node is shown. Cloud computing node 10 is merely one example of a suitable cloud computing node and is not intended to suggest any limitation as to the scope of use or functionality of the embodiments of the invention described herein. Nevertheless, cloud computing node 10 is capable of implementing and / or performing any of the functionality described above.

[0016] Cloud computing node 10 may include portable electronic devices such as computer systems / servers 12 or communication devices that are operable with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with computer systems / servers 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable appliances, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0017] The computer system / server 12 may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system / server 12 may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0018] 1, computer system / server 12 in cloud computing node 10 is shown in the form of a general purpose computing device. Components of computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 coupling various system components including system memory 28 to processor 16.

[0019] Bus 18 represents any one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a high-speed graphics port, and a processor or local bus using any of a variety of bus architectures, including, by way of example and not limitation, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0020] Computer system / server 12 typically includes a variety of computer system readable media. Such media may be any available media that can be accessed by computer system / server 12 and includes both volatile and nonvolatile media, removable and non-removable media.

[0021] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be provided for reading from and writing to a non-removable non-volatile magnetic medium (not shown, and typically referred to as a "hard drive"). Although not shown, a magnetic disk drive may be provided for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), as well as an optical disk drive for reading from or writing to a removable non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical media. In such instances, each may be connected to the bus 18 by one or more data media interfaces. As further shown and described below, the memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of an embodiment of the present invention.

[0022] A program / utility 40 having a set (at least one) of program modules 42 may be stored in memory 28, as well as, by way of example and not limitation, an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or any combination thereof, may include an implementation of a networked environment. The program modules 42 generally perform the functions and / or methods of embodiments of the present invention described herein.

[0023] The computer system / server 12 may also communicate with one or more external devices 14, such as a keyboard, pointing device, display 24, one or more devices that allow a user to interact with the computer system / server 12, or any device that allows the computer system / server 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.), or combinations thereof. Such communication may occur through an input / output (I / O) interface 22. Additionally, the computer system / server 12 may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or combinations thereof, through a network adapter 20. As shown, the network adapter 20 communicates with other components of the computer system / server 12 through a bus 18. Although not shown, it is understood that other hardware and / or software components may be used in conjunction with the computer system / server 12. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0024] Referring now to FIG. 2, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or cell phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof, may communicate. The nodes 10 may communicate with each other. They may be physically or virtually grouped in one or more networks, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or combinations thereof, as described above (not shown). This allows the cloud computing environment 50 to offer infrastructure, platform, and / or software as a service without the cloud consumer having to maintain resources on the local computing device. It should be understood that the types of computing devices 54A-N shown in FIG. 2 are intended to be exemplary only, and that the computing node 10 and cloud computing environment 50 may communicate with any type of computerized device via any type of network and / or network addressable connections (e.g., using a web browser).

[0025] Referring now to Figure 3, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 2) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 3 are intended to be merely exemplary, and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0026] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, Reduced Instruction Set Computer (RISC) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0027] The virtualization layer 70 provides an abstraction layer over which the following examples of virtual entities may be provided: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0028] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing and other resources used to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides protection for data and other resources as well as identity verification for cloud consumers and tasks. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides cloud computing resource allocation and management so that requested service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-positioning and procurement of cloud computing resources where future requirements are anticipated according to SLAs.

[0029] The workload layer 90 provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functionality that may be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instructional delivery 93, data analytics processing 94, transaction processing 95, and natural language expression variant generation 96. The functionality of natural language expression variant generation 96 is described in the following embodiments of the present disclosure.

[0030] When training or testing a model, several semantically equivalent sentences or expressions may be necessary. For example, an intent may be expressed in various natural language expressions, so that a large number of intent variants are generally essential for building an intent recognition model. For example, the intent of "how to treat gastritis" may be expressed using different variants such as "what secrets should gastritis be treated with?", "I have gastritis, what do you do?", "how do you treat gastritis?", "how to cure gastritis", etc. The more variants used in model building, the more accurate the model will be.

[0031] Therefore, there is a need to generate variants for natural language expressions in natural language processing applications. Conventional solutions require manual preparation of sufficient variants for an expression. However, collecting high-quality variants is a labor-intensive, time-consuming and error-prone activity. Thus, it is desirable to provide an automatic mechanism for generating variants.

[0032] According to an embodiment of the present invention, a solution for automatic generation of natural language expression variants is provided. In this solution, key entities of a source expression are replaced by predefined symbols to obtain a structured expression. One or more template structured expressions are prepared and available for selection, each of the template structured expressions containing identical predefined symbols for representing key entities. If the one or more template structured expressions are similar to the structured expression of the source expression, e.g., have a similarity higher than a threshold similarity, the one or more template structured expressions are selected to generate a variant for the source expression. In particular, the variant for the source expression is generated by replacing predefined symbols in the one or more selected template structured expressions with key entities in the source expression. Through this solution, some variants can be obtained automatically, thereby achieving improved efficiency and accuracy compared to manual preparation.

[0033] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Embodiments of the present invention will now be described with reference to the accompanying drawings. Reference is first made to Figure 4, which illustrates a block diagram of an example system 400 for generating natural language expression variants, in accordance with an embodiment of the present invention. System 400 includes a number of modules for performing a number of stages in generating one or more variant expressions for a source expression.

[0034] It will be understood that the modules of system 400 shown in FIG. 4 may be implemented as one or more software engines, components, etc., which are comprised of logic for performing the functionality ascribed to a particular module. Each module may be implemented with one or more such software engines, components, etc. The software engines, components, etc. execute on one or more processors of one or more computing systems or devices and use or operate on data stored in one or more storage devices, memories, etc. on the one or more computing systems.

[0035] In some embodiments, different modules of system 400 may be implemented as a single module, or a single module of system 400 may be separated into more than one module. In some embodiments, one or more additional modules may be included in system 400. It should be understood that the expressions shown in FIG. 4 and described below are provided as specific examples for illustrative purposes only. Embodiments of the present invention may be applied to process any other natural language expressions.

[0036] As shown, the system 400 includes an entity recognition module 410 configured to obtain a source representation 402 in a natural language and identify / recognize key entities 412 (referred to as "source key entities") within the source representation 402. The source representation 402 may be in the form of any natural language and may be composed of one or more words, characters, or other linguistic units. The representation 402 may be a sentence, a phrase, a text string, or the like. There are no limitations on the length and specific content of the source representation 402 to be processed. As it is anticipated in this disclosure to generate one or more variations on the source representation 402, the source representation 402 may also be referred to as a seed representation or simply a seed. In the illustrated example, the source representation 402 is the English phrase "Basic symptoms of gastritis."

[0037] The source key entity 412 may be represented using one or more words, characters, or other linguistic units contained in the source representation 402. A key entity in a representation may be a core entity for the intent of the representation, and the core entity may be a terminology, a named entity, or any other important word / character in a field in the source representation. The source key entity 412 is a key entity in the source representation 402. In the example shown, the source key entity "gastritis" 412 is recognized from the example source representation 402. In some embodiments, more than one source key entity 412 may be recognized from the source representation 402. Various entity recognition techniques may be applied to identify the source key entity 412. Some example embodiments for key entity recognition are described in detail below with reference to FIG. 5.

[0038] The source key entity 412 and the source representation 402 are provided to a structured construction module 420 included in the system 400. The structured construction module 420 is configured to modify the source representation 402 into a structured representation 422 by replacing the source key entity 412 in the source representation 402 with a predetermined symbol. For illustrative purposes only, the predetermined symbol is "EN" in the example of FIG. 4, and then the source representation 402 can be modified to be the structured representation 422 "Basic Symptom of EN". The predetermined symbol may be any common symbol that is also used to construct other structured representations. To avoid potential confusion, the predetermined symbol may be selected as a text symbol, a computer symbol, an abbreviation, or any other type of representation, or a combination thereof, that may rarely occur in natural language expressions. There is no particular restriction on the symbol for replacing the key entity.

[0039] In embodiments in which more than one source key entity 412 is recognized from the source representation 402, the same predetermined symbol may be used to replace those source key entities (e.g., if the key entities are in a cooperative relationship), or different predetermined symbols may be used to replace the different source key entities.

[0040] System 400 also includes a template recognition module 430 configured to determine whether there are one or more available template structured representations similar to structured representation 422 and to select the similar template structured representations for generating a variant for source representation 402. The set of template structured representations may be stored in a template database 405 accessible to template recognition module 430. The set of template structured representations may also be referred to as a library of template structured representations. The template structured representations may be collected from various sources or corpora and may each be processed to include a predefined symbol (e.g., “EN”) to represent one or more key entities contained therein.

[0041] In natural language, there is a relatively fixed sentence structure that can be used to construct various expressions, but the specific key entities included in those expressions may vary. By pre-processing available natural language expressions to ignore key entities with predefined meaningless symbols, the structure of the expressions can be used as a template to generate different forms of expressions. For example, by searching from web pages, databases, dictionaries, or other sources, or a combination of them, several available expressions such as "What are the symptoms of chronic pharyngitis", "What are the signs of influenza", and "What are the cardinal symptoms of inflammatory bowel disease" are discovered. The template structured expressions can be obtained by replacing the key entities in those expressions such as "chronic pharyngitis", "influenza", and "inflammatory bowel disease" with a predefined symbol such as "EN".

[0042] To find similar template structured representations for structured representation 422, template recognition module 430 determines the respective similarities between structured representation 422 and each template structured representation stored in template database 405. The similarities between two structured representations can be determined in various ways used to measure the similarity between normal sentences / expressions, except that certain symbols contained in the structured representations can be considered to be identical. In some embodiments, the structural similarities between structured representation 422 and template structured representation are considered. The structures of those representations become more obvious after replacing key entities contained therein with identical symbols. In some embodiments, the meanings of key words / phrases contained in structured representation 422 and template structured representation can also be considered to ensure that similar structured representations contain identical or similar key words / phrases.

[0043] The template recognition module 430 may compare each calculated similarity with a threshold similarity (for convenience of explanation, referred to as a "first threshold similarity"). The first threshold similarity may be set according to the actual requirements for the final transformation. If one or more of the similarities exceed the first threshold similarity, the corresponding template structured representation may be selected to generate a transformation for the source representation 402. In the example of FIG. 4, as shown, N template structured representations are selected from the template database 405, including at least template structured representation 432-1 "What are the symptoms of EN", template structured representation 432-2 "What are the signs of EN", and template structured representation 432-N "What are the basic symptoms of EN". The selected template structured representations 431-1, 432-2, ..., 432-N similar to the structured representation 422 may be referred to as template structured representations 432, collectively or individually.

[0044] In some embodiments, to further reduce computational delay, the template structured representations stored in the template database 405 may first be classified or clustered into groups, each group including one or more similar template structured representations. Any clustering or classification method, such as a k-means algorithm, a k-means++ algorithm, a density-based clustering method, etc., may be applicable to classify the template structured representations.

[0045] Upon receiving the structured representation 422 of the source representation 402, the template recognition module 430 may calculate a sparse similarity between the structured representation 422 and one of the template structured representations of each group (such as at the cluster centroid), and select one or more groups having a higher similarity with the structured representation 422 (top K groups, K is 1 or more). In this way, most of the dissimilar template structured representations may be filtered out to reduce computational consumption. The template recognition module 430 may further calculate a fine similarity between the structured representation 422 and each template structured representation included in the selected group. The fine similarity may then be compared to a first threshold similarity to select a similar template structured representation 432 for the structured representation 422.

[0046] The similar template structured representations 432 are provided to a variant generation module 440 included in the system 400. The variant generation module 440 is configured to generate one or more variant representations for the source representation 402 by replacing a predetermined symbol in at least one template structured representation 432 with a source key entity 412. As an example, by replacing the predetermined symbol "EN" with the source key entity "gastritis" 412 determined from the source representation 402, the template structured representations 432-1 to 432-N are modified to be variant representation 442-1 "What are the symptoms of gastritis", variant representation 442-2 "What are the signs of gastritis", and variant representation 442-N "What are the basic symptoms of gastritis". They are collectively or individually referred to as variant representations 442. Because the structured representations are similar and the exact source key entities for source representation 402 are embedded in the similar template structured representation, the generated transformed representation 442 may have substantially the same intent as source representation 402 and may be semantically equivalent or similar to source representation 402.

[0047] It should be understood that the template structured representation may be in the same natural language of the source representation 402. If it is expected to generate variants for some source representation in a different natural language than the template structured representation, the source representation may first be translated into a representation in that language, for example by a language translation model, and the final variants may be back-translated into the language of the source representation.

[0048] 5 illustrates a block diagram of an example of an entity recognition module 410 of the system 400 in accordance with an embodiment of the present invention. In this example, the entity recognition module 410 applies a trained machine learning model to identify source key entities 412 from the source representation 402.

[0049] In entity recognition, the source representation 402 may be converted into tokens. Each token is a consecutive sequence of characters grouped together, which may be a subword, a word, a punctuation mark, or a part of speech such as (but not limited to) a comma, a hyphen, a slash, or an apostrophe. In some embodiments, each token may be the smallest unit within a representation. For example, the source representation 402 "basic symptoms of gastritis" may be tokenized as "basic", "symptoms", "of", and "gastritis", where each word is a token.

[0050] The entity recognition module 410 includes an embedding generator 510 for vectorizing the source representation 402 or tokens of the source representation 402 with their respective embeddings 512, each of which includes a vector of a certain degree. Vectorization or embedding is commonly used in natural language processing. There are various available methods that may be used to obtain an embedding of an expression. In an embodiment, the embedding generator 510 may randomly assign a numeric value within a certain range (e.g., -1 to 1) to each token of the source representation 402. It should be understood that other embeddings may also be possible.

[0051] The embeddings 512 of the source representations 402 are provided as inputs to a machine learning model 520 for entity recognition to recognize the source key entities 412. The machine learning model 520 is pre-trained with a training dataset. The machine learning model 520 may be any kind of machine learning model or deep learning model for entity recognition, some examples of which may include a bidirectional long short-term memory (Bi-LSTM) model, a conditional random field (CRF) model, a Bi-LSTM-CRF model, and any other kind of models or neural networks. Such models are widely used in natural language processing, and their working principles are known to those skilled in the art.

[0052] The machine learning model 520 may be trained for entity recognition for a specific domain application by using a training data set related to this specific domain, or may be trained as a generic key entity recognition model for applications in different domains by using training data sets related to all of those domains. By using the trained machine learning model 520, it is possible to identify the source key entities 412 in a more efficient and accurate manner.

[0053] Although entity recognition in the source representation 402 has been described, the same or similar recognition methods may be applied to recognize key entities when preparing the template structured representation in the template database 405.

[0054] In the above-described embodiment, one or more template structured representations 432 having a similarity higher than a first threshold similarity may be selected from the template database 405 to generate a variant representation for the source representation 402. In some cases, there may be no or a small number of template structured representations 432 having a similarity higher than the first threshold similarity in the template database 405. To provide sufficient variant representations, in some embodiments, the system 400 may be configured to add more template structured representations in the template database 405 in this case. FIG. 6 shows a block diagram of an example of the system 400 in such an embodiment. For clarity, example representations at different stages of processing are not shown in FIG. 6.

[0055] As shown, the system 400 further includes a template addition module 610. The template addition module 610 is configured to search for one or more additional template structured representations 612 based on the source representation 402 and then add the one or more additional template structured representations 612 into the template database 405 for future use. The template structured representations may be found from any given corpus, including books, dictionaries 601, the Internet 602, etc. By dynamically triggering the addition of template structured representations, the template database 405 may be enriched on demand during use of the system 400.

[0056] If it is determined that the number of similar template structured representations (those having a similarity with the structured representation 422 higher than the first threshold similarity) for the structured representation 422 is less than a threshold number, the template adding module 610 may be triggered by the template recognition module 430. The threshold number may be configurable and may be set as a value indicating a minimum requirement for the number of variant representations for the source representation 402. For example, if there are no similar template structured representations or only a few template structured representations for the structured representation 422, the template adding module 610 may be triggered to find and add more template structured representations 612 based on the source representation 402.

[0057] 7 shows a block diagram of an embodiment of the template addition module 610 in the system of FIG. 6. The template addition module 610 includes a search engine 710 for searching one or more corpora 601, 602 for expressions similar to the source expression 402. In performing the search, the search engine 710 may calculate a semantic similarity between the source expression 402 and the expressions included in the corpus. The search engine 710 may retrieve one or more raw expressions in the same natural language as the source expression 402. The raw expressions have a relatively high similarity to the source expression 402 and typically include at least the source key entity 412 (the word "gastritis" in the above embodiment).

[0058] As some examples, the raw representations retrieved by search engine 710 may include representation 712-1 "What are the symptoms of gastritis," representation 712-2 "What are the signs of gastritis," representation 712-3 "What are the symptoms of having gastritis," and representation 712-M "What are the basic symptoms of gastritis." Raw representations 712-1, 712-2, 712-3, ..., 712-M may be collectively or individually referred to as raw representations 712.

[0059] The template structured representation 612 may be determined from those raw representations 712. In some embodiments, when the scale of the corpora 601, 602 is large, a large amount of similar raw representations may be obtained, some of which may have only slight differences. To reduce memory consumption and avoid generating too many redundant variations of the source representations, in some embodiments, some of the raw representations may be filtered out to construct the template structured representation.

[0060] Specifically, the template addition module 610 includes a clustering-based selection module 720 configured to cluster the raw representations 712 into a plurality of groups of raw representations and then select a raw representation based on the clustering results. Each of the plurality of groups includes at least one of the raw representations 712. Any clustering or classification method, such as a k-means algorithm, a k-means++ algorithm, a density-based clustering method, etc., may be applicable for clustering. During clustering, the raw representations may be converted into vectors to enable similarity calculation. The clustering-based selection module 720 may select each raw representation from the plurality of groups of raw representations. One raw representation may be selected from each of the groups, such as the one at the center or centroid of the group. For example, raw representations 712-1, 712-2, and 712-M are selected, while other similar raw representations, such as raw representation 712-3, are discarded.

[0061] The clustering-based selection module 720 then generates a number of further template structured representations by replacing the source key entities 412 in the selected raw representations 712 with predefined symbols. As an example, the template structured representations include template structured representation 612-1 "What are the symptoms of EN?", template structured representation 612-2 "What are the symptoms of EN?", and template structured representation 612-N "What are the cardinal symptoms of EN?". It should be noted that those template structured representations 612 are inferred not to be included in the template database 405 when the template recognition module 430 performs a template search against the database.

[0062] In some embodiments, the template addition module 610 may further provide one or more of the raw representations 712, particularly those raw representations selected to generate the template structured representation 612, as at least one variant representation of the source representation 402. Thus, not only does the template database 405 include more templates, but more variants of the source representation 402 may be provided for future use.

[0063] 8 shows a flowchart of an example method 800 according to an embodiment of the present disclosure. The method 800 may be implemented in the system 400 as shown in FIG. 4. For illustrative purposes, the method 800 will be described with reference to FIG.

[0064] In block 810, the system 400 determines a structured representation for the source expression in natural language by replacing source key entities in the source expression with predefined symbols. In block 820, the system 400 selects at least one template structured representation from the set of template structured representations based on respective similarities between the structured representation and each template structured representation in the set. Each template structured representation includes predefined symbols for representing key entities. In block 830, the system 400 generates at least one variant representation for the source expression by replacing predefined symbols in the at least one template structured representation with source key entities.

[0065] In some embodiments, selecting the at least one template structured representation includes comparing the respective similarities to a first threshold similarity, and selecting the at least one template structured representation in response to a similarity of the respective similarities between the structured representation and at least one of the respective template structured representations in the set exceeding the first threshold similarity.

[0066] In some embodiments, the method 800 further includes determining a number of template structured representations having a similarity to the structured representation greater than a first threshold similarity, and in response to the determined number being less than the threshold number, obtaining, from the predetermined corpus, at least one raw expression in the natural language having a similarity to the source representation greater than a second similarity threshold, where the at least one expression includes a source key entity, and generating at least one further template structured representation based on the at least one raw expression.

[0067] In some embodiments, a plurality of raw representations are obtained and generating at least one further template structured representation includes clustering the plurality of raw representations into a plurality of groups of raw representations, each of the plurality of groups including at least one of the plurality of raw representations, selecting a respective raw representation from the plurality of groups of raw representations, and generating the plurality of further template structured representations by replacing source key entities in the selected raw representations with predefined symbols.

[0068] In some embodiments, the method 800 further includes providing at least one of the selected raw representations as at least one transformed representation for the source representation.

[0069] In some embodiments, the respective similarity measures are based at least in part on structural similarity measures between the structured representation and each template structured representation in the set.

[0070] In some embodiments, the method 800 further includes identifying source key entities from the source representation based on a trained machine learning model for entity recognition.

[0071] The invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0072] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves that allow instructions to be recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being inherently ephemeral signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through electrical wires.

[0073] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0074] The computer readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer readable program instructions may execute completely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by individualizing the electronic circuitry with state information of the computer readable program instructions to perform aspects of the invention.

[0075] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0076] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to manufacture a machine, such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium that may instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium having instructions stored thereon comprises an article of manufacture including instructions for performing aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0077] Computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to produce a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0078] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may in fact be implemented as one step, executed simultaneously, substantially simultaneously, partially or fully overlapping in time, or the blocks may be executed in the reverse order depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0079] The description of various embodiments of the present invention is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used in this specification has been selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for generating natural language expression variants, comprising: determining, by one or more processors, a structured representation for a source expression in a natural language by replacing source key entities in said source expression with predefined symbols; selecting, by the one or more processors, at least one template structured representation from a set of template structured representations based on a respective degree of similarity between the structured representation and each template structured representation in the set, each of the set of template structured representations including the predetermined symbol for representing a key entity; generating, by the one or more processors, at least one variant representation of the source representation by replacing the predetermined symbols in the at least one selected template structured representation with the source key entities; determining, by the one or more processors, a number of template structured representations that have a similarity to the structured representation that is greater than a first threshold similarity; in response to the determined number being less than a threshold number, obtaining, by one or more processors, from a predetermined corpus, at least one raw expression in the natural language having a similarity to the source expression greater than a second similarity threshold, the at least one raw expression including the source key entity; generating, by the one or more processors, at least one further template structured representation based on the at least one raw representation; A method comprising:

2. Selecting the at least one template structured representation comprises: comparing, by the one or more processors, the respective similarity measures to the first threshold similarity measure; selecting, by one or more processors, in response to a measure of similarity between the structured representation and at least one of the respective template structured representations in the set exceeding the first threshold similarity measure; The method of claim 1 , comprising:

3. A plurality of raw representations are obtained, and generating the at least one further template structured representation comprises: clustering, by the one or more processors, the plurality of raw representations into a plurality of groups of raw representations, each of the plurality of groups including at least one of the plurality of raw representations; selecting, by the one or more processors, a respective raw representation from a plurality of groups of raw representations; generating, by the one or more processors, a plurality of further template structured representations by replacing the source key entities in selected of the raw representations with the predetermined symbols; The method of claim 1 , comprising:

4. The method of claim 3 , further comprising providing, by the one or more processors, at least one of the selected raw representations as at least one transformed representation of the source representation.

5. The method of claim 1 , wherein the respective similarity measures are based at least in part on structural similarity measures between the structured representation and the respective template structured representations in the set.

6. 10. The method of claim 1, further comprising identifying, by the one or more processors, the source key entities from the source representation based on a trained machine learning model for entity recognition.

7. 1. A system comprising: A processing unit; a memory coupled to the processing unit and storing instructions, the instructions, when executed by the processing unit, determining a structured representation for a source expression in a natural language by replacing source key entities in the source expression with predefined symbols; selecting at least one template structured representation from a set of template structured representations based on a respective degree of similarity between the structured representation and each template structured representation in the set, each of the set of template structured representations including the predetermined symbol for representing a key entity; generating at least one variant representation of the source representation by replacing the predetermined symbols in the at least one selected template structured representation with the source key entities; determining a number of template structured representations having a similarity to the structured representation greater than a first threshold similarity; in response to the determined number being less than a threshold number, obtaining, from a predefined corpus, at least one raw expression in the natural language having a similarity to the source expression greater than a second similarity threshold, the at least one raw expression including the source key entity; generating at least one further template structured representation based on the at least one raw representation; the memory performing operations including: A system comprising:

8. Selecting the at least one template structured representation comprises: comparing each of the similarities to the first threshold similarity; selecting the at least one template structured representation in response to a measure of similarity between the structured representation and at least one of the respective template structured representations in the set exceeding the first threshold similarity measure; The system of claim 7 , comprising:

9. A plurality of raw representations are obtained, and generating the at least one further template structured representation comprises: clustering the plurality of raw representations into a plurality of groups of raw representations, each of the plurality of groups including at least one of the plurality of raw representations; selecting a respective raw representation from said plurality of groups of raw representations; generating a plurality of further template structured representations by replacing the source key entities in selected of the raw representations with the predetermined symbols; The system of claim 7 , comprising:

10. The system of claim 9 , wherein the operations further comprise providing at least one of the selected raw representations as at least one transformed representation for the source representation.

11. The system of claim 7 , wherein the respective similarity measures are based at least in part on structural similarity between the structured representation and the respective template structured representation in the set.

12. 8. The system of claim 7, wherein the operations further comprise identifying the source key entities from the source representation based on a trained machine learning model for entity recognition.

13. A computer program product which, when run on a computer, causes the computer to carry out the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Question answering system, data retrieval method, and computer program

    JP2006252380A

  • Approximate Template Matching for Natural Language Queries

    JP2019525272A

  • Question answering system, data search method, and computer program

    US20060206472A1

  • Approximate template matching for natural language queries

    US20170364520A1

  • Method and apparatus for natural language query in a workspace analytics system

    US20180060297A1