Method, apparatus, device, medium and product for generating training data

Abstract

Disclosed are methods and apparatuses, devices, media and products for generating training data. The method includes aggregating user intent by a first language model based on a first indicator among multiple indicators. The method also includes determining a first user question by the first language model based on the user intent. The method also includes generating a first domain-specific language (DSL) answer by the first language model based on the first user question. The method also includes constructing a second DSL answer based on a second indicator among multiple indicators and an element corresponding to a DSL structure. The method also includes generating a second user question by the first language model based on the second DSL answer. The method also includes training a second language model based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have a DSL structure. In this way, the NL2DSL capability and generalization of the model can be greatly improved without or with a small amount of manually annotated data.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of computers, and in particular, to methods and apparatuses, devices, media, and products for generating training data. Background Art

[0002] In recent years, language models have developed rapidly and have become an influential and highly-regarded technology. As a representative of language models, large language models (LLMs) are widely used for their powerful natural language processing and deep learning capabilities.

[0003] One of the core functions of language models is to significantly simplify the previously complex and cumbersome knowledge acquisition process, providing a new way for rapid retrieval, understanding and application of information. With the technical empowerment of language models, users can overcome the high knowledge barriers of the past and achieve functions equivalent to those difficult-to-use professional tools, thereby reducing the entry threshold and learning costs of knowledge. Summary of the invention

[0004] An embodiment of the present disclosure provides a solution for generating training data.

[0005] In a first aspect of the present disclosure, a method for generating training data is provided, the method comprising aggregating user intent by a first language model based on a first indicator among multiple indicators. The method further comprises determining a first user question by the first language model based on the aggregated user intent. The method further comprises generating a first domain-specific language (DSL) answer by the first language model based on the determined first user question. The method further comprises constructing a second DSL answer based on a second indicator among the multiple indicators and at least one element corresponding to a DSL structure. The method further comprises generating a second user question by the first language model based on the constructed second DSL answer. The method further comprises training a second language model based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have a DSL structure.

[0006] In a second aspect of the present disclosure, a device for generating training data is provided, the device comprising an intent aggregation module configured to aggregate user intents by a first language model based on a first indicator among multiple indicators. The device further comprises a first question determination module configured to determine a first user question by a first language model based on the aggregated user intent. The device further comprises a first answer generation module configured to generate a first domain-specific language (DSL) answer by a first language model based on the determined first user question. The device further comprises a second answer construction module configured to construct a second DSL answer based on a second indicator among multiple indicators and at least one element corresponding to a DSL structure. The device further comprises a second question generation module configured to generate a second user question by a first language model based on the constructed second DSL answer. The device further comprises a training module configured to train a second language model based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have a DSL structure.

[0007] According to a third aspect of the present disclosure, an electronic device is provided. The computing device includes a processor and a memory, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the processor executes a method or process according to an embodiment of the present disclosure.

[0008] According to a fourth aspect of the present disclosure, a machine-readable storage medium is provided, wherein machine-executable instructions are stored on the machine-readable storage medium, and when the machine-executable instructions are executed by a processor, the processor is enabled to execute a method or process according to an embodiment of the present disclosure.

[0009] In a fifth aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer-readable storage medium and includes a computer program, which, when executed by a processor of a computer, causes the processor to perform a method or process according to an embodiment of the present disclosure.

[0010] Please note that the summary is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The summary is not intended to identify key features or essential features of the disclosure, nor is it intended to limit the scope of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other purposes, features and advantages of the present disclosure will become more clearly understood by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, in which:

[0012] Figure 1is a diagram schematically illustrating an example environment in which methods and / or processes according to embodiments of the present disclosure may be implemented;

[0013] Figure 2 is a flowchart schematically illustrating a method for generating training data according to an embodiment of the present disclosure;

[0014] Figure 3 A diagram schematically illustrates an example process 300 of training data generation according to an embodiment of the present disclosure;

[0015] Figure 4 is a diagram schematically illustrating an example process 400 of forward data enhancement according to an embodiment of the present disclosure;

[0016] Figure 5 is a diagram schematically illustrating an example process 500 of reverse data enhancement according to an embodiment of the present disclosure;

[0017] Figure 6 is a diagram schematically illustrating an apparatus for generating training data according to an embodiment of the present disclosure;

[0018] Figure 7 is a schematic block diagram of an example device that may be used to implement embodiments according to the present disclosure.

[0019] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0021] In the description of the embodiments of the present disclosure, the term "including" and its variations should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly indicated to be different.

[0022] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0023] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0024] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0025] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0026] As mentioned above, language models significantly simplify the process of knowledge acquisition and reduce the entry threshold and learning cost of knowledge. Data query and analysis methods based on language models can provide a relatively simplified solution, allowing non-technical or professional users to easily obtain and understand data and make better decisions.

[0027] NL2DSL (Natural Language to Domain Specific Language), a technology that converts natural language to domain-specific language, makes data query more intuitive, simple, and efficient. However, the training of language models for NL2DSL relies on a large amount of manual annotation, which may lead to poor model generalization due to the limitations of the annotators' thinking and the high homogeneity of the annotated data. In addition, the DSL structure is usually associated with specific usage scenarios (for example, business lines, etc.), so a certain amount of appropriate training data is required to fine-tune the model to make it targeted. Without using a large amount of relevant training data for training, the results output by the model may be degraded.

[0028] In order to solve at least some of the above and other potential problems, an embodiment of the present disclosure proposes a scheme for generating training data. The scheme includes aggregating user intent by a first language model based on a first indicator among multiple indicators. The scheme also includes determining a first user question by the first language model based on the aggregated user intent. The scheme also includes generating a first domain-specific language (DSL) answer by the first language model based on the determined first user question. The scheme also includes constructing a second DSL answer based on a second indicator among the multiple indicators and at least one element corresponding to the DSL structure. The scheme also includes generating a second user question by the first language model based on the constructed second DSL answer. The scheme also includes training a second language model based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have a DSL structure.

[0029] In this way, by performing positive data augmentation and negative data augmentation, the generalization performance of the model can be improved while improving the model's semantic understanding and indicator selection capabilities, so that the NL2DSL capability and generalization of the model can be greatly improved in the absence of or a small amount of manually labeled data. The generated training data covers a variety of situations from simple to complex, achieving comprehensive coverage of DSL structures.

[0030] Reference below Figures 1 to 7 It should be understood that these exemplary embodiments are provided only to enable those skilled in the art to better understand and implement the embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure in any way.

[0031] Figure 1 1 is a schematic diagram schematically illustrating an example environment 100 in which methods and / or processes according to embodiments of the present disclosure may be implemented. Figure 1 As shown in FIG. 1 , environment 100 includes a local computing node 110 and computing nodes 121 and 122 arranged in the cloud. Figure 1 The implementation environment of the embodiment of the present disclosure is described by taking a hybrid computing system as an example. It should be understood that this is only an exemplary and non-restrictive description, and other different types of computing systems are also feasible, such as distributed or centralized computing systems. The appropriate computing environment configuration can be selected according to actual use needs.

[0032] exist Figure 1In the embodiment, only limited components and exemplary connection relationships are shown. It should be understood that this is for the purpose of ease of explanation and ease of illustration and is not intended to limit the scope of the present disclosure, and other different components may also exist. For example, a display component or an audio input component, etc. In an exemplary and non-limiting manner, a user can ask questions or inquiries through a language description, such as typing a language description of the question or query via a corresponding interface component in a graphical user interface (GUI) displayed on a display component, or, for example, speaking a language description of the question or query via an audio input component such as a microphone.

[0033] Multiple computing nodes with computing capabilities can process inputs from users, and the processed results will be output to users by multiple computing nodes. In an exemplary but non-restrictive manner, computing nodes 121 and 122 can be arranged locally in the user, while computing nodes 121 and 122 can be arranged in the cloud. The cloud can indicate a service model built based on cloud or distributed technology. In this mode, computing resources, storage resources, etc. are coupled together through a network to form a schedulable and scalable resource collection. At least a portion of these resources can be dynamically accessed or allocated to complete tasks without the need to locally own or manage these physical resources.

[0034] A computing node may be a device with computing capabilities. For example, a computing node may be provided with a processor and a memory, etc., or may be equipped with a dedicated accelerator (such as a graphics processing unit (GPU)). In addition, a computing node may store and maintain data. The data (e.g., the indicators used and the corresponding data, etc.) and models (e.g., the teacher model used and the student model to be trained, etc.) involved in the method for generating training data according to an embodiment of the present disclosure may be centrally arranged on one node, or may be distributedly deployed on some of the multiple nodes.

[0035] like Figure 1 As shown in the example, the local computing node 110 and the computing nodes 121 and 122 in the cloud can be interconnected to communicate with each other through a network. Figure 1 Taking the architecture in as an example, computing nodes 110, 121 and 122 can communicate via a network to achieve, for example, data synchronization or sharing between nodes. Multiple computing nodes in environment 100 can process computing tasks in parallel. In addition, multiple computing nodes in environment 100 can have redundancy and fault tolerance mechanisms to ensure reliable execution of computing tasks. When a computing node fails, the system can automatically migrate the job to other nodes that are working normally to ensure the continuity and availability of the task.

[0036] Examples of computing nodes may include supercomputers, personal computers, laptop computers, vehicle-mounted computing devices, mobile devices (such as smart phones, tablet computers, etc.), wearable electronic devices, multimedia devices, personal digital assistants (PDAs), smart home devices, consumer electronic products, or a combination of any one or more of the above devices, etc. It should be understood that the computing nodes described herein are merely exemplary and non-restrictive, and for example, other different types of computing nodes may also be used.

[0037] Combined with the above Figure 1 Describes an example environment in which the methods and / or processes according to embodiments of the present disclosure may be implemented. Figure 2 The method 200 for generating training data according to an embodiment of the present disclosure is described. Through the method 200, the NL2DSL capability and generalization of the model can be greatly improved without or with a small amount of manually labeled data.

[0038] Figure 2 is a flowchart schematically illustrating a method 200 for generating training data according to an embodiment of the present disclosure. At 210, based on a first indicator among multiple indicators, a first language model aggregates user intent. According to an embodiment of the present disclosure, multiple indicators are determined, and these indicators are associated with specific scenarios (e.g., business scenarios) to be used for data query and analysis. The indicator indicates one of the various aspects of the scenario and has a mapping relationship with the corresponding data. For example, in an online shopping scenario, the indicator may include the number of payees, refund rate, etc. Using natural language processing technology, the first language model identifies potential or possible user intent from the first indicator (one or more) among the multiple indicators. In some embodiments, the first language model may be a teacher model.

[0039] At 220, based on the aggregated user intent, the first language model determines the first user question. According to an embodiment of the present disclosure, based on the user intent aggregated at 210, the first language model extracts and analyzes key information, topics, etc., and converts the aggregated user intent into structured or semi-structured data through lexical analysis, syntactic analysis, semantic understanding, etc. The first user question (one or more) is determined based on the structured or semi-structured data associated with the user intent. Here, the corresponding answer will be found based on the question to improve the semantic understanding, indicator selection ability, etc. of the model during training.

[0040] At 230, the method further includes generating a first domain-specific language (DSL) answer by a first language model based on the determined first user question. The generation of the first DSL answer (one or more) from the first user question by the first language model may refer to a certain amount of text data, a knowledge graph, a knowledge base in a specific domain, etc. The first language model may generate an answer with coherence, accuracy, and readability through language generation techniques (such as sequence-to-sequence models, attention mechanisms, etc.) according to the context of the question, domain-specific terms and concepts, and user needs. The DSL-based answer has a predetermined DSL structure (i.e., an organizational form) to facilitate further use in subsequent processes.

[0041] At 240, a second DSL answer is constructed based on a second indicator among the multiple indicators and at least one element corresponding to the DSL structure. According to an embodiment of the present disclosure, one or more elements may be used to indicate a desired DSL structure. In other words, it may be used to construct a desired DSL question-answering architecture. Based on the second indicator (one or more) among the multiple indicators and the element corresponding to the DSL structure, a second DSL answer (one or more) may be constructed. Here, the corresponding question is found from the answer, aiming to improve the generalization performance of the model during training, etc.

[0042] At 250, based on the constructed second DSL answer, the first language model generates a second user question. According to an embodiment of the present disclosure, based on the second DSL answer constructed at 240, the first language model extracts information, analyzes the answer, etc., and generates a second user question (one or more) corresponding to the second DSL answer by means of its natural language processing capability, deep learning capability, etc. In the following, forward data augmentation and reverse data augmentation according to an embodiment of the present disclosure will be described in further detail.

[0043] At 260, a second language model is trained based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have a DSL structure. According to an embodiment of the present disclosure, the model is trained using both forward-generated training data and reverse-generated training data, so that a large amount of suitable training data can be obtained while getting rid of the thinking limitations and high homogeneity of manual annotation. Training based on such training data can promote the NL2DSL capability and generalization of the model. In some embodiments, the second language model can be a student model.

[0044] According to the method 200 for generating training data in an embodiment of the present disclosure, by performing forward data augmentation and reverse data augmentation, the generalization performance of the model can be improved while improving the semantic understanding and indicator selection capabilities of the model, so that the NL2DSL capability and generalization of the model are greatly improved in the absence of or a small amount of manually labeled data. The generated training data covers various situations from simple to complex, achieving comprehensive coverage of DSL structures.

[0045] Figure 3 FIG. 3 schematically illustrates an example process 300 of training data generation according to an embodiment of the present disclosure. Figure 3 The definition of the structure or organization of the DSL content (i.e., Figure 3 The DSL definition 310 in FIG. 3 may be based on multiple indicators, such as an indicator set 320. The indicator set 320 may include one or more indicators that meet business security requirements (i.e., compliance indicators 322). Here, business security requirements may refer to business confidentiality or privacy that meets the requirements of an enterprise or organization.

[0046] In the case where the indicators in the indicator set 320 are insufficient, they can be supplemented. In some embodiments, the number of multiple indicators in the indicator set 320 is determined. In response to the number of indicators being less than the number threshold, a business indicator 324 can be generated based on at least one business line. When generating the business indicator 324, the business indicator 324 can be desensitized according to the business security requirements to obtain a desensitized indicator 326, so that the indicator used for supplementation meets the business security requirements. The desensitized indicator 326 will be provided as a further supplement to the indicator set 320. It should be understood that the indicator set 320 may also include other different indicators besides those shown in the figure.

[0047] According to an embodiment of the present disclosure, forward generation 330 may be performed based on a portion (i.e., the first indicator) of the indicator set 320, and reverse generation 340 may be performed based on another portion (i.e., the second indicator) of the indicator set 320. It should be understood that this is not restrictive, and the same one or more indicators are also feasible.

[0048] In the forward generation 330, at the intent aggregation 332, the user intent may be aggregated by the first language model based on the first indicator. Based on the aggregated user intent, at the question generation 334, a first user question may be generated by the first language model. Based on the generated first question, a first DSL answer may be generated by the first language model. In some embodiments, a plurality of DSL answers may be obtained by inputting the first user question into the first language model twice or more. Then, the difference between the plurality of DSL answers obtained may be determined. If the determined difference is less than or equal to a difference threshold, it may be determined that the generation process is stable and the generated DSL answer is in line with expectations. When the determined difference is greater than the difference threshold, the generation process needs to be adjusted, for example, the architecture, parameters, etc. of the first language model are updated.

[0049] In reverse generation 340, at DSL construction 342, a second DSL answer may be constructed based on the second indicator and at least one element corresponding to the DSL structure, and such construction may be based on field feature sampling. Based on the constructed second DSL answer, at question generation 344, a second user question may be generated by the first language model. In some embodiments, at question rewriting 346, the generated first user may be rewritten to meet the natural language description requirement, which may be achieved by a natural language descriptive threshold. Figure 4 and Figure 5 Further details of forward generation 330 and reverse generation 340 are described further.

[0050] Figure 4 FIG. 4 is a diagram schematically illustrating an example process 400 of forward data enhancement according to an embodiment of the present disclosure. Taking an online shopping scenario as an example, Figure 4 As shown in , user intent 420, such as “refund details”, can be aggregated based on a first indicator 410. The first indicator 410 may include one or more candidate fields, such as refund amount, dissatisfaction rate, and the like.

[0051] According to an embodiment of the present disclosure, the relevant fields 410 may be determined based on the aggregated user intent, such as "refund amount, number of refunded people, and number of refunded orders". Next, the first user question 450 may be generated based on the determined relevant fields 410 and a time element indicating a time interval, such as "Learn more about today's refund situation". In some embodiments, the time interval for obtaining data may be limited by a start time and an end time, such as Figure 4 As shown in .

[0052] Figure 5 is a diagram schematically illustrating an example process 500 of reverse data enhancement according to an embodiment of the present disclosure. Figure 5The second indicator 502 and the elements corresponding to the predetermined DSL structure are shown. Figure 5 As shown in , there is a dependency relationship between them. Examples of elements corresponding to the predetermined DSL structure may include group 504, index condition 506, dimension condition 508, number 510, sort 512, and time 514. It should be understood that these elements are only exemplary and non-restrictive, and other different elements may also be included.

[0053] In some embodiments, the metric 502 may include its name and the corresponding metric function (such as year-on-year, quarter-on-quarter, etc.). The group 504 may indicate the grouping dimension used in the query, such as "view based on a certain dimension" or "group based on a certain dimension". The metric condition (where) 506 may indicate the retrieval condition for the metric when querying, which includes an operator (Operator) that identifies the relationship between multiple conditions, such as "AND" and "OR". The filter condition (Filter) may be represented by a field (Field), an operator (Operator) and a value (Value).

[0054] Dimension condition 508 may indicate the retrieval condition for the dimension when querying, which includes similar operators and filter conditions. Limit 510 may indicate the number of records to be intercepted, and all are returned by default. Offset may indicate the starting position of the interception, and Count may indicate the number of intercepted records (cannot be 0). Order 512 may indicate the sorting intention of the user's query. Time 514 may indicate the index of the time interval to be queried.

[0055] According to an embodiment of the present disclosure, the complexity of the second indicator 502 and at least one element can be determined by performing field feature sampling on the second indicator 502 and at least one element. Business fields are not completely independent, but the contents of each field are generated in sequence according to the dependency relationship. Each field has a predefined sampling number range, and the field feature refers to the number of feature sampling. Figure 5 As shown in , the second indicator 502 and the complexity 530 of each element can be determined by the sampling number range 520. Complexity refers to the cumulative value of field features, which identifies the complexity of generating DSL. Setting the complexity can control the overall generated data not to differ too much from the actual distribution and ensure that the special cases of each field can be covered. The second DSL answer can be generated based on the second indicator 502 and the complexity 530 of each element.

[0056] According to an embodiment of the present disclosure, a second user question can be generated based on the second DSL answer and the question-answering pattern from the time question-answering database. The introduction of the question-answering pattern of the time question-answering database can facilitate machine recognition of time limits, such as "Double 12". The reversely generated data DSL structure has comprehensive coverage, and the generated questions cover a variety of situations from simple to complex.

[0057] Figure 6 Schematically illustrates an apparatus 600 for generating training data according to an embodiment of the present disclosure, and the apparatus 600 may include multiple units or modules for performing the steps or actions in the method or process discussed above. Figure 6 As shown in , the apparatus 600 includes: an intent aggregation module, configured to aggregate user intents through a first language model based on a first indicator among multiple indicators; a first question determination module, configured to determine a first user question through the first language model based on the aggregated user intent; a first answer generation module, configured to generate a first domain-specific language (DSL) answer through the first language model based on the determined first user question; a second answer construction module, configured to construct a second DSL answer based on a second indicator among the multiple indicators and at least one element corresponding to a DSL structure; a second question generation module, configured to generate a second user question through the first language model based on the constructed second DSL answer; and a training module, configured to train a second language model based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have the DSL structure.

[0058] In some embodiments, the DSL structure can be determined based on the multiple indicators, and the device 600 can also include an indicator module configured to: determine the number of the multiple indicators; in response to determining that the number of the multiple indicators is less than a number threshold, generate a business indicator based on at least one business line; and provide the business indicator.

[0059] In some embodiments, the first question determination module may be further configured to: determine relevant fields from the first indicator based on the aggregated user intent; and generate the first user question based on the determined relevant fields and a time element indicating a time interval.

[0060] In some embodiments, a self-checking module may be further included, configured to: obtain a plurality of DSL answers by inputting the first user question into the first language model twice or more; and determine differences between the obtained plurality of DSL answers.

[0061] In some embodiments, each of the multiple indicators may include a corresponding name and an indicator function, and the at least one element may include at least one of the following: a group element; an indicator condition element; a dimension condition element; a number element; a sorting element; and a time element.

[0062] In some embodiments, the second answer construction module may be further configured to: determine the complexity of the second indicator and the at least one element by performing field feature sampling on the second indicator and the at least one element; and generate the second DSL answer based on the second indicator and the complexity of the at least one element.

[0063] In some embodiments, the second question generation module may be further configured to generate the second user question based on the second DSL answer and a question-answering pattern from a temporal question-answering database, wherein the second user question is rewritten to meet a natural language descriptiveness threshold.

[0064] In some embodiments, the first language model may be a teacher model and the second language model may be a student model.

[0065] Figure 7 FIG. 1 is a block diagram of an electronic device 700 according to some embodiments of the present disclosure. The device 700 may be a device or apparatus described in an embodiment of the present disclosure. Figure 7 As shown, the device 700 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU / GPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704. Although not shown in FIG. Figure 7 As shown in FIG. 7 , device 700 may further include a coprocessor.

[0066] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0067] The various methods or processes described above may be performed by the CPU / GPU 701. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU / GPU 701, one or more steps or actions in the methods or processes described above may be performed.

[0068] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0069] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0070] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0071] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0072] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0073] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0074] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0075] Various embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the various embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

1. A method for generating training data, comprising: Aggregating user intent by a first language model based on a first indicator among the multiple indicators; Based on the aggregated user intent, the first language model determines a first user question according to relevant fields and a time element indicating a time interval; Based on the determined first user question, generating a first domain specific language (DSL) answer by the first language model; Constructing a second DSL answer based on a second indicator among the multiple indicators and at least one element corresponding to the DSL structure, comprising: Determine the complexity of the second indicator and the at least one element by performing field feature sampling on the second indicator and the at least one element, and generating the second DSL answer based on the second indicator and the complexity of the at least one element; Based on the constructed second DSL answer, generating a second user question by the first language model; and A second language model is trained based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have the DSL structure.

2. The method according to claim 1, wherein the DSL structure is determined based on the plurality of indicators, the method further comprising: determining the number of the plurality of indicators; In response to determining that the number of the plurality of indicators is less than a number threshold, generating a business indicator based on at least one business line; as well as The business indicator is provided.

3. The method according to claim 1, wherein: The relevant fields are determined from the first metric based on the aggregated user intent.

4. The method according to claim 1, further comprising: obtaining a plurality of DSL answers by inputting the first user question into the first language model two or more times; as well as Differences between the plurality of DSL answers obtained are determined.

5. The method according to claim 1, wherein each of the plurality of indicators comprises a corresponding name and an indicator function, and the at least one element comprises at least one of the following: Group elements; Indicator condition element; Dimension condition element; Number elements; sorting elements; and Time element.

6. The method of claim 1, wherein generating the second user question comprises: generating the second user question based on the second DSL answer and the question-answering pattern from the temporal question-answering database, The second user question is rewritten to meet the natural language description requirement.

7. The method according to claim 1, wherein: The first language model is a teacher model, and The second language model is a student model.

8. An apparatus for generating training data, comprising: an intention aggregation module, configured to aggregate user intentions through a first language model based on a first indicator among the multiple indicators; A first question determination module is configured to determine a first user question based on the aggregated user intent and according to the first language model according to relevant fields and a time element indicating a time interval; A first answer generation module, configured to generate a first domain specific language (DSL) answer by using the first language model based on the determined first user question; The second answer constructing module is configured to construct a second DSL answer based on a second indicator among the multiple indicators and at least one element corresponding to the DSL structure, including: Determine the complexity of the second indicator and the at least one element by performing field feature sampling on the second indicator and the at least one element, and generating the second DSL answer based on the second indicator and the complexity of the at least one element; A second question generating module is configured to generate a second user question through the first language model based on the constructed second DSL answer; and The training module is configured to train a second language model based on the first user question and the first DSL answer and the second user question and the second DSL answer, wherein the first DSL answer and the second DSL answer have the DSL structure.

9. An electronic device, comprising: processor; as well as A memory coupled to the processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the processor is caused to perform the method according to any one of claims 1 to 7.

10. A machine-readable storage medium having machine-executable instructions stored thereon, wherein the machine-executable instructions, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program which, when executed by a processor of a computer, causes the processor to perform the method according to any one of claims 1 to 7.

Images