Language model training and atomic operation arrangement method and device and readable medium

Abstract

The invention provides a language model training and atomic operation arrangement method and device and a readable medium, and belongs to the technical field of communication. The language model training method comprises the following steps: acquiring a universal pre-training language model and communication scene sample data, and pre-training the universal pre-training language model to obtain a base model; training the base model by using the communication scene sample data to obtain a language model for atomic operation arrangement; wherein the communication scene sample data comprises a mapping relationship between the natural language description information of the communication scene sample and the description information of the corresponding atomic operation chain sample. The method is used for realizing intellectualization of atomic operation arrangement driven by a scene function.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a language model training and atomic operation arrangement method, device and readable medium. Background Art

[0002] The existing "scenario atomic operation orchestration" usually requires business scenario experts to configure each specific scenario to generate related operation chains, that is, manually specify a series of atomic operations required by the business scenario, and then reuse them according to the scenario name.

[0003] Every time a new scenario is added in "Scenario Atomic Operation Orchestration", it is necessary for business scenario experts to perform operation orchestration and testing. In actual use, users can only select specific scenarios from the existing scenario pool, and the selection space is extremely limited. It can be seen that the traditional "Scenario Atomic Operation Orchestration" product has low flexibility, poor scalability, and poor user experience. Summary of the invention

[0004] The present disclosure provides a language model training and atomic operation arrangement method, device and readable medium.

[0005] The first aspect of the present disclosure provides a language model training method, comprising:

[0006] Obtaining a general pre-trained language model and communication scenario sample data, and performing pre-training using the general pre-trained language model to obtain a base model;

[0007] The base model is trained using the communication scenario sample data to obtain a language model for atomic operation orchestration; wherein the communication scenario sample data includes a mapping relationship between natural language description information of the communication scenario sample and description information of the corresponding atomic operation chain sample.

[0008] A second aspect of the present disclosure provides an atomic operation arrangement method, including:

[0009] Obtain natural language description information of communication scenario requirements;

[0010] Using a language model to analyze the natural language description information of the communication scenario requirements to obtain description information of the atomic operation chain corresponding to the communication scenario requirements; wherein the language model is trained using the method described in the first aspect;

[0011] According to the description information of the atomic operation chain, at least one atomic operation constituting the atomic operation chain is obtained and arranged.

[0012] A third aspect of the present disclosure provides a language model training device, comprising:

[0013] An acquisition module, used to acquire a general pre-trained language model and communication scenario sample data, and pre-train the general pre-trained language model to obtain a base model;

[0014] A training module is used to train the base model using the communication scenario sample data to obtain a language model for atomic operation orchestration; wherein the communication scenario sample data includes a mapping relationship between the natural language description information of the communication scenario sample and the description information of the corresponding atomic operation chain sample.

[0015] A fourth aspect of the present disclosure provides an atomic operation arrangement device, including:

[0016] An acquisition module, used to acquire natural language description information of communication scenario requirements;

[0017] An analysis module, used to analyze the natural language description information of the communication scenario requirements using a language model to obtain description information of the atomic operation chain corresponding to the communication scenario requirements; wherein the language model is trained using the method described in any one of claims 1 to 5;

[0018] The orchestration module is used to obtain and orchestrate at least one atomic operation constituting the atomic operation chain according to the description information of the atomic operation chain.

[0019] A fifth aspect of the present disclosure provides an electronic device, including:

[0020] at least one processor;

[0021] A memory having at least one program stored thereon, wherein when the at least one program is executed by the at least one processor, the at least one processor implements the method according to the first aspect or the second aspect;

[0022] At least one I / O interface is connected between the processor and the memory and is configured to implement information interaction between the processor and the memory.

[0023] A sixth aspect of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to the first aspect or the second aspect is implemented.

[0024] The present disclosure has the following advantages:

[0025] After pre-training the general pre-trained language model, communication scenario sample data is continued to be used for training. The communication scenario sample data includes the mapping relationship between the natural language description information of the communication scenario samples and the description information of the corresponding atomic operation chain samples, so that the trained language model can be used for atomic operation orchestration. Because the language model is trained based on the mapping relationship between the natural language description information of the communication scenario samples and the description information of the corresponding atomic operation chain samples, the language model can realize the conversion from communication scenarios to atomic operation chains, thereby avoiding the situation where business scenario experts are required to perform operation orchestration and testing for each new scenario, providing a basis for realizing the intelligentization of scenario-driven atomic operation orchestration, reducing expert maintenance costs, and the interaction method based on natural language description information also greatly enhances the flexibility of use and reduces the difficulty of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flow chart of a language model training method provided in an embodiment of the present disclosure;

[0027] Figure 2a A schematic diagram of the dimensions of a two-stage joint training provided in an embodiment of the present disclosure;

[0028] Figure 2b A schematic diagram of a training process of a language model for atomic operation orchestration provided in an embodiment of the present disclosure;

[0029] Figure 3 A schematic diagram of a flow chart of an atomic operation arrangement method provided in an embodiment of the present disclosure;

[0030] Figure 4 A schematic diagram of the structure of a language model training device provided in an embodiment of the present disclosure;

[0031] Figure 5 A schematic diagram of the structure of an atomic operation arrangement device provided in an embodiment of the present disclosure;

[0032] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0034] As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0035] The terms used in the present disclosure are only used to describe specific embodiments and are not intended to limit the present disclosure.As used in the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0036] When the terms “comprising” and / or “made of…” are used in the present disclosure, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0037] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meaning as those commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined in this disclosure.

[0038] The embodiments of the present disclosure provide a language model training method, which can be applied to any electronic device, such as a terminal or a server, whose hardware and software environment can support the training of the language model.

[0039] Figure 1 The figure is a flow chart of the language model training method provided by the embodiment of the present disclosure. The language model training process mainly includes the following steps:

[0040] Step 101, obtain a general pre-trained language model and communication scenario sample data, and pre-train the general pre-trained language model to obtain a base model.

[0041] Among them, the general pre-trained language model can be any language model of natural language processing (Natural Language Processing, NLP), and the specific model used is not limited here.

[0042] In some embodiments, the pre-training of the universal pre-trained language model to obtain a base model includes: performing unidirectional autoregressive pre-training of the universal pre-trained language model using communication field corpus to obtain a base model.

[0043] Scenario-driven atomic operation orchestration involves considerable professional knowledge. By using a large amount of communication domain corpus to pre-train the general pre-trained language model, the joint probability with the communication domain can be enhanced, thereby avoiding the problem that the general pre-trained language model is not competent for joint probability modeling in the communication domain.

[0044] Exemplarily, a unidirectional language model architecture (Decoder-only) is used to perform the first phase of training on the general pre-trained language model. The first phase training task is to cyclically predict the next word (token). The communication field corpus used includes but is not limited to:

[0045] (1) Documents describing various services in the communications field (actual production requirements, background, technologies involved, possible solutions, etc.);

[0046] (2) Instructions and operation manuals for various products (released or under development) in the communications field;

[0047] (3) Documentation on the capabilities of various network elements in the communications field (capabilities of each physical or logical object actually involved);

[0048] (4) Interface documentation for services in the communications field (all services provided by the company internally or externally);

[0049] (5) Operation and maintenance logs and documents in the communications field (accumulated in the R&D backend);

[0050] (6) Log documents of the engineering service end in the communications field (accumulation of external operation and maintenance);

[0051] (7) All documents on business topics pointed to by the common document interface that is open internally in the field of communications (internal search);

[0052] (8) A detailed description document of each atomic operation in the communication domain;

[0053] (9) Documents on various proper nouns in the field of communications (special to a specific field).

[0054] In order to ensure the effect of pre-training, the original communication field corpus is pre-processed to improve the text quality. Among them, some documents that have passed the publishing process are of high quality and can be used only after special character filtering or replacement, proper noun processing and other operations. All remaining documents pass the quality filtering process, especially all internal public documents crawled through the intranet. Quality filtering methods include at least one of the following:

[0055] (1) Delete documents with blank content, such as documents with only large and small titles, serial numbers, etc.;

[0056] (2) Delete text that is not long enough;

[0057] (3) Deleting text irrelevant to the business: a text classifier can be trained by combining some documents with irrelevant documents obtained from outside. The text classifier is used to make preliminary judgments and then manually calibrated. The calibrated text classifier is used to delete text irrelevant to the business in most documents after being tested and annotated.

[0058] (4) Organize some standard or semi-standard format documents into reasonable descriptive documents through language;

[0059] (5) Short documents on proper nouns and numerical explanations need to be reasonably expanded, such as adding descriptions of scene segments where proper nouns are used, referential replacement, etc.

[0060] (6) Delete repeatedly nested documents, that is, documents that only contain links to other documents.

[0061] All documents that have been quality filtered can be used for autoregressive pre-training. The language model after this training will serve as the base model for adjusting the subsequent series of tasks.

[0062] Step 102: Use the communication scenario sample data to train the base model to obtain a language model for atomic operation orchestration; wherein the communication scenario sample data includes a mapping relationship between natural language description information of the communication scenario sample and description information of the corresponding atomic operation chain sample.

[0063] In some embodiments, a two-step prompt mode is used to continue training the base model, with targeted decomposition instructions.

[0064] Specifically, the using the communication scenario sample data to train the base model to obtain a language model for atomic operation orchestration includes: using the communication scenario sample data to perform first-stage prompt training on the base model to obtain a first-stage language model; using the communication scenario sample data to perform second-stage prompt training on the first-stage language model to obtain a second-stage language model; performing instruction pattern training on the second-stage language model to obtain a language model for atomic operation orchestration.

[0065] In some embodiments, the communication scenario sample data is used to perform first-stage prompt training on the base model to obtain a first-stage language model, including: obtaining a first type of prompt sample data; wherein the first type of prompt sample data is obtained by annotating the natural language description information of the communication scenario samples using a first type of prompt template, and the first type of prompt template is used to perform vertical analysis on the natural language description information of the communication scenario samples to prompt the association between scenarios, phenomena and atomic operations; and the base model is trained using the first type of prompt sample data to obtain a first-stage language model.

[0066] Among them, vertical analysis refers to the disassembly of the natural language description of a single communication scenario sample, and the integration of the scenes, phenomena and atomic operations directly obtained from the disassembly according to the first type of prompt template.

[0067] In some embodiments, obtaining the first type of prompt sample data includes: obtaining natural language description information corresponding to a single data link from the natural language description information of the communication scenario sample; using the first type of prompt template to disassemble the natural language description information corresponding to the single data link from a preset dimension to obtain the first type of prompt sample data corresponding to the single data link, wherein the preset dimension includes at least one of the following: a sequential dimension of atomic operations, and a sequential dimension of scenarios and operations.

[0068] By disassembling a single data chain, we can obtain the first type of prompt sample data, so that the trained model can disassemble and record more fine-grained knowledge points. By disassembling the natural language description information corresponding to the single data chain from the two dimensions of the order dimension of atomic operations and the order dimension of scenarios and operations, we can guide the model to focus on more fine-grained knowledge points and pay attention to the data chain from two dimensions. Through the order dimension of atomic operations, we can guide the model to pay attention to the business meaning and business process under the operation name, so that the model can pay attention to the areas, terminals, domains, physical objects, data objects, etc. involved in atomic operations. Through the order dimension of scenarios and operations, we can guide the model to pay attention to the relationship between scenarios and certain operations, rather than just a complete atomic operation chain.

[0069] In the first stage of prompt training, the first type of prompt sample data is used to train the base model. The first type of prompt sample data points to specific NLP tasks, such as the relationship judgment task of whether the scene and atomic operation are included, and the order judgment task between atomic operations in a specific scene. The <scenario-atomic operation chain> described by each communication scenario sample is processed according to the first type of prompt template to generate the corresponding first type of prompt sample data. The base model is trained with the first type of prompt sample data to strengthen the direction of knowledge points learned by the model. For example, the <scenario S: atomic operation A, atomic operation B, atomic operation C, ..., atomic operation N> described by the communication scenario sample is generated according to the first type of prompt template to generate the corresponding first type of prompt sample data. Table 1 shows some examples of the first type of prompt template:

[0070] Table 1

[0071]

[0072]

[0073] Among them, in the first category prompt sample data corresponding to the first category prompt template, the natural language description word (such as token) corresponding to the answer has two position modes: 1) the answer is at the end of the text; 2) the answer is in the text. When constructing the first category prompt sample data, the answer is mainly at the end of the text. If a <scenario-atomic operation chain> has K descriptions of the scene and the length of the atomic operation chain is N, then the original data can be expanded by combining atomic operations to obtain more first category prompt sample data. An original data can be expanded to at least C(N, i) represents the number of combinations of i atomic operations selected from N atomic operations, and the first type of prompt sample data can be further randomly enhanced, for example, by adding A(N, i) represents the number of permutations of selecting i atomic operations from N atomic operations. It can be seen that the first type of prompt sample data can be enriched by permuting and combining the atomic operations included in the <scenario-atomic operation chain> of a communication scenario sample description.

[0074] The first type of prompt sample data in the first stage is the vertical knowledge decomposition and mining of a single <scenario-atomic operation chain>.

[0075] In some embodiments, the communication scenario sample data is used to perform second-stage prompt training on the first-stage language model to obtain the second-stage language model, including: obtaining second-category prompt sample data; wherein the second-category prompt sample data is obtained by annotating the natural language description information of the communication scenario samples using the second-category prompt template, and the second-category prompt template is used to perform horizontal analysis on the natural language description information of the communication scenario samples to prompt the extension between scenarios, phenomena and atomic operations; and the second-category prompt sample data is used to train the first-stage language model to obtain the second-stage language model.

[0076] Among them, horizontal analysis refers to the disassembly of the natural language description of the communication scenario samples, and the expansion of the scenes, phenomena and atomic operations directly obtained from the disassembly according to the second type of prompt templates.

[0077] In some embodiments, obtaining the second type of prompt sample data includes:

[0078] Using the natural language description information of the communication scenario samples, obtaining the phenomenon corresponding to each scenario, using the phenomenon corresponding to each scenario, determining the intersection between the scenario and the phenomenon, and using the intersection between the scenario and the phenomenon to generate the second type of prompt sample data;

[0079] and / or

[0080] Using the natural language description information of the communication scenario sample to obtain each atomic operation and the business meaning corresponding to each atomic operation, using the business meaning corresponding to each of the atomic operations to determine the intersection between the atomic operations, and using the intersection between the atomic operations to generate the second type of prompt sample data;

[0081] and / or

[0082] The natural language description information of the communication scenario samples is used to obtain the phenomena corresponding to each scenario, the atomic operations corresponding to the scenario, and the business meanings corresponding to each atomic operation. The phenomena corresponding to each scenario, the atomic operations corresponding to the scenario, and the business meanings corresponding to each atomic operation are used to determine the intersections among the scenarios, phenomena, and atomic operations. The intersections among the scenarios, phenomena, and atomic operations are used to generate the second type of prompt sample data.

[0083] The second type of prompt sample data obtained based on the intersection of the three parts can learn the association of phenomena in the field behind the scene through the cross-guidance model between scenes and phenomena, can learn the business association in the field behind the operation through the cross-guidance model between atomic operations, and can learn the complete orchestration mode through the cross-guidance model between scenes, phenomena and atomic operations. It can be seen that the essence of training the reasoning ability of the language model is to provide the model with the corresponding "thinking" mode through the natural language description information of the communication scene samples. The natural language description information of the communication scene samples does not need to provide any explicit and clear relationship between scenes and atomic operations, but cross-learns different <scenes-phenomenon-atomic operations> through the prompt engineering guidance model. The analysis is as follows:

[0084] For the intersection between scenarios and phenomena, the first-stage language model obtained by pre-training the pre-trained language model and the first-stage prompt training already has the basic knowledge of the arrangement scenarios in the field, that is, it can know the meaning in the field represented by each scenario and the phenomena in the field; the phenomenon is the most accumulated and richly described corpus in the communication field, and the rigorous description is manifested as the change of indicators in various dimensions, such as XXX KPI is reduced to 50%, and the inductive description is manifested as the user-level experience, such as the signal has deteriorated; the first-stage language model is further cross-learned through the in-field phenomena of the scenario, that is, the scenario is first grouped into multiple groups, each of which represents a certain common phenomenon in the field, such as <weak coverage, cross coverage> has some similar KPI indicator performance, then the second type of prompt sample data is completed through the common phenomena in the field between the scenarios, guiding the model to think about the connection between the in-field phenomena behind the scenario;

[0085] For the intersections between atomic operations, the model is guided to think through the domain business knowledge implied by the atomic operations. For example, different areas, domains, and terminals have different atomic operations, and different data objects have different atomic operations. The atomic operations are grouped so that the atomic operations in each group have certain domain business commonalities and connections. Then, the second type of prompt sample data is prepared through the commonalities and connections of the business knowledge implied behind the atomic operations, guiding the model to think about the domain business connections behind the atomic operations.

[0086] For the intersection among scenes, phenomena and atomic operations, this part of cross-learning is the combination of the intersection between scenes and phenomena and the intersection between atomic operations, such as Figure 2a The figure shows a dimensional diagram of the two-stage joint training, which is to connect the phenomena in the field between scenes with the business implied by atomic operations. The core grouping logic is <common phenomena between scenes, related operations between scenes>. By producing such second-class prompt sample data, the model is guided to connect the thinking of the two-stage prompt training, thereby guiding the model to learn the knowledge of the complete orchestration model.

[0087] In the second stage of prompt training, the second type of prompt sample data is used to continue training the first stage language model. The second type of prompt sample data is a horizontal reasoning extension of scenes and atomic operations, that is, horizontal parsing. Table 2 shows some examples of the second type of prompt templates in the second stage of prompt training:

[0088] Table 2

[0089]

[0090]

[0091] In some embodiments, the instruction pattern training of the second-stage language model to obtain a language model for atomic operation orchestration includes: obtaining instruction pattern sample data; wherein the instruction pattern sample data is obtained by annotating natural language description information of a communication scenario sample using an instruction template, and one sample data of the instruction pattern sample data includes an instruction task text and a corresponding annotated answer, and the instruction task text is used to ask questions about the relationship between scenarios, phenomena and atomic operations; the instruction pattern sample data is used to train the second-stage language model to obtain a language model for atomic operation orchestration.

[0092] Exemplarily, some optional instruction modes are as follows:

[0093] (1) Give all the atomic operations required in scenario S and the order of the atomic operations;

[0094] (2) Give the two atomic operations following the atomic operation D in scenario S;

[0095] (3) Select all atomic operations that can support scenario S from the listed multiple operations;

[0096] (4) sorting the listed multiple atomic operations required to support scenario S;

[0097] (5) Give a brief description of the atomic operation C;

[0098] (6) Summarize the common points of multiple atomic operations;

[0099] (7) Summarize the commonalities of the given multiple scenarios.

[0100] Through supervised training with command pattern sample data, the downstream task intent is further aligned, minimizing the gap between the model's data training mode and the user's actual usage mode. The focus of command pattern training is on training the ability to follow commands. According to the user's usage commands, the command pattern can be directly summarized into generation, classification, extraction, question-and-answer, and other types of modes.

[0101] According to the association between the scene and the atomic operation chain, the scene description information, and the atomic operation description information included in the natural language description information of the communication scene sample, the instruction task text is manually written and annotated answers are given, that is, the instruction pattern sample data is obtained; after the second-stage language model is used to generate the answer corresponding to the instruction task text, the answer is compared with the annotated answer, and the second-stage language model is further calibrated according to the comparison result. The second-stage language model that has been further calibrated with the instruction pattern sample data can be used for scenario-driven atomic operation orchestration.

[0102] In some embodiments, in the process of training the base model with the communication scenario sample data to obtain the language model for atomic operation arrangement, the atomic operation is identified from the natural language description information of the communication scenario sample based on the pre-built atomic operation pool. The atomic operation pool includes each atomic operation in the communication field and the name corresponding to the atomic operation, and the name includes the description keyword of the corresponding atomic operation.

[0103] Exemplarily, there are as many as 300 atomic operations described in a standardized manner in the atomic operation pool. During the training process of the language model for atomic operation orchestration, 20% of the data chains of atomic operations that have been orchestrated through scenario association are used for training, so that the obtained language model can understand the data chain and perform reasoning based on background knowledge in the communication field, and extend the ability of induction, comparison, and association during reasoning to un-orchestrated atomic operations, and at the same time be able to correctly follow the user's instructions to perform operation orchestration tasks.

[0104] In an exemplary embodiment, Figure 2b The figure shows a schematic diagram of the training process of the language model for atomic operation orchestration, which mainly includes:

[0105] Step 201, constructing an atomic operation pool;

[0106] Step 202, determining whether a pre-trained model in the communication field, i.e., a base model, already exists, if yes, executing step 204, otherwise, executing step 203;

[0107] Step 203, using communication domain corpus to train a general pre-training model to obtain a communication domain pre-training model;

[0108] Step 204, continue training the pre-trained model in the communication field using a two-step prompting mode;

[0109] Step 205, continue to perform instruction mode training on the model obtained by the two-step prompt model training to obtain the final language model.

[0110] In the disclosed embodiment, after pre-training the general pre-trained language model, communication scenario sample data is further used for training, wherein the communication scenario sample data includes a mapping relationship between natural language description information of the communication scenario samples and description information of the corresponding atomic operation chain samples, so that the trained language model can be used for atomic operation orchestration. Because the language model is trained based on the mapping relationship between the natural language description information of the communication scenario samples and the description information of the corresponding atomic operation chain samples, the language model can realize the conversion from the communication scenario to the atomic operation chain, thereby avoiding the situation where business scenario experts are required to perform operation orchestration and testing for each new scenario, providing a basis for realizing the intelligentization of scenario-driven atomic operation orchestration, reducing expert maintenance costs, and the interaction method based on natural language description information also greatly enhances the flexibility of use and reduces the difficulty of use.

[0111] An atomic operation orchestration method is also provided in an embodiment of the present disclosure, which uses the language model obtained in the above training process to perform atomic operation orchestration. The method can be applied to any electronic device, which can be a terminal or a server, etc.

[0112] Figure 3 The figure is a schematic diagram of the atomic operation arrangement method provided by the embodiment of the present disclosure. The atomic operation arrangement method mainly includes the following steps:

[0113] Step 301: Obtain natural language description information of communication scenario requirements.

[0114] Step 302: Analyze the natural language description information of the communication scenario requirements using a language model to obtain the description information of the atomic operation chain corresponding to the communication scenario requirements;

[0115] Among them, the language model is trained using the language model training method described in the above embodiments. For the detailed training process, refer to the language model training process described above and will not be repeated here.

[0116] Step 303: Obtain at least one atomic operation that makes up the atomic operation chain according to the description information of the atomic operation chain and perform orchestration.

[0117] In some embodiments, the description information of the atomic operation chain includes at least one description keyword of the atomic operation that makes up the atomic operation chain, and the arrangement order of the description keywords of the at least one atomic operation;

[0118] The obtaining at least one atomic operation that makes up the atomic operation chain according to the description information of the atomic operation chain and performing orchestration includes: obtaining the corresponding atomic operations from the atomic operation pool according to the description keywords of the at least one atomic operation included in the atomic operation chain; among them, the atomic operation pool includes various atomic operations in the communication field and the names corresponding to the atomic operations, and the names include the description keywords corresponding to the atomic operations; orchestrate the atomic operations obtained from the atomic operation pool according to the arrangement order of the description keywords of the at least one atomic operation that makes up the atomic operation chain.

[0119] In some embodiments, the atomic operation pool includes at least one of the following:

[0120] Data - type atomic operations, and the names of the data - type atomic operations include at least the description keywords of the data object and the description keywords of the operation;

[0121] Task - type atomic operations, and the names of the task - type atomic operations include at least the description keywords of the operation and the description keywords of the communication scenario.

[0122] Among them, the atomic operation core is a piece of code method with a single function that can be reused. These methods are necessary operations in many communication - field scenarios. For example, in the large - scale scenario of base - station upgrade, the necessary reusable atomic operations include: <Cell data backup>, <DV data backup>, <Alarm data backup>, etc.

[0123] In the process of building the atomic operation pool, in order to use natural language processing related technologies to replace the hard-coded calls of code scripts or artificial component configurations, a unified naming rule design is performed on the atomic operations at the bottom layer. The design enables the naming of atomic operations to include potential object business dependencies, hierarchical relationships, object types in the communication field and clarify the operation functions. Exemplarily, the name of the data class atomic operation is directly marked by abstracting <(area) + (end) + (domain) + (physical object) + data object + operation>, wherein the name of the data class atomic operation includes at least the name of the data object and the name of the atomic operation, and when at least one of the area name, end name, domain name or physical object name of the data class atomic operation can be obtained, it is included in the name of the data class atomic operation; the name of the task class atomic operation is marked by <operation + communication scenario>, and the name of the task class atomic operation includes the name of the atomic operation and the name of the communication scenario. Such a naming rule can accurately and clearly describe the function of the atomic operation, while containing potential business logic.

[0124] For example, the <area> in the data-type atomic operation mainly includes <existing network> and <planning> in the communication network management; <existing network> means that the corresponding set of configurations or parameters have been issued to the actual physical unit, and <planning> means that all the corresponding configurations or parameters are not yet in the system library, and the actual physical unit configuration has not been updated. <existing network> to <planning> implies the complete business logic of "issuing". <end> in the communication network management mainly refers to <network planning>, <network optimization>, <network management>, etc., representing different business endpoints. Different business endpoints involve different data structures and data operation methods in the entire operation process, and also contain potential communication business flow logic directions, that is, <network planning> to <network optimization> and then to <network management>. <domain> generally represents the standard category of physical objects, such as time division duplexing technology (TDD), network element type (such as ITRAN), etc.

[0125] In task-type atomic operations, for example, the "collection task" in <create collection task> refers to a special sub-scenario, which specifically refers to the network element status and network element alarm information. Task-type atomic operations can be understood as simply encapsulated data operations of a higher level.

[0126] By continuously accumulating at least hundreds of atomic operations in the communication field through naming standards and reasonable encapsulation, these atomic operations can form an atomic operation pool for upper-level intention combination calls. By naming the atomic operation, the name can reflect the potential business logic information of the atomic operation, and express the business logic of the atomic operation in a concise natural language, so as to facilitate the subsequent acquisition of the corresponding atomic operation from the atomic operation pool according to the description information of the atomic operation, and facilitate the call through natural language technology.

[0127] In the exemplary embodiment, the user inputs a natural language description of a specific intention, and the language model for atomic operation arrangement maps it to an intrinsic scenario through semantic understanding, and generates the corresponding atomic operation chain in the correct order among more than 300 atomic operations in the atomic operation pool through reasoning ability. Among them, "semantic understanding" refers to the understanding of the user's true intention, which can be simply understood as a classification task, that is, classifying the user's intention input into a specific scenario.

[0128] In principle, the user input is regarded as x, and the language model's arrangement task is to generate the most likely y based on the input x, that is, In NLP, y refers to a string of characters (tokens), including single words and single symbols. In the arrangement task, this string of tokens is connected to form an atomic operation chain. The maximum probability here does not refer to the usual semantic dimension. For example, "Beijing", "Beijing", and "North" are all reasonable and coherent in the usual semantic dimension, but in the sentence "The capital of China is XX", it can only be "Beijing", that is, P('Beijing'|'The capital of China is') has the highest probability. Then the highest probability refers to the highest probability of "string co-occurrence" under the atomic operation arrangement of the communication field scenario, which represents the specific knowledge point under this constraint. For example, the value of P('8) License file loading'|'7) License file preparation') is close to 1, which represents the knowledge point that: in the license file operation process, file loading is required immediately after file preparation, and there are generally no other operations in between.

[0129] In one exemplary embodiment, the user enters:

[0130] "The current XX site version does not match that of all nearby sites. Recommended operation instructions are given."

[0131] or:

[0132] "It is detected that the supporting version of a certain base station is not the latest release. Please provide operational suggestions for version upgrade."

[0133] System output:

[0134] "Recommended scenario <Base station version change>, detailed steps are as follows:

[0135] 1) Obtain the target version;

[0136] 2) Confirm the version change authority;

[0137] 3) The target version enters the database;

[0138] 4) Confirm the version storage result;

[0139] 5) Pre-check of version change data;

[0140] 6) Preparation of License files;

[0141] 7) Loading of License files;

[0142] 8) Detection of network element version data;

[0143] 9) Backup of configuration data;

[0144] 10) Backup of cell status data;

[0145] 11) Backup of alarm data;

[0146] 12) Creation of version change tasks;

[0147] 13) Download of target version;

[0148] 14) Execution of version change tasks;

[0149] 15) Confirmation of the completion status of version change tasks”.

[0150] Among them, the tokens are “push”, “recommend”, “field”, “scene”, “<”, “base”, “station”, “version”, “change”, “>”, “,”, “detailed”, “steps”, “are” “as” “follows:” “1)” “target” “version” “acquisition” “2)” “confirmation”......

[0151] In the embodiments of the present disclosure, after pre-training a general pre-trained language model, training is continued using communication scenario sample data. The communication scenario sample data includes the mapping relationship between the natural language description information of the communication scenario sample and the description information of the corresponding atomic operation chain sample, so that the trained language model can be used for atomic operation orchestration. Since the language model is trained based on the mapping relationship between the natural language description information of the communication scenario sample and the description information of the corresponding atomic operation chain sample, the language model can realize the conversion from the communication scenario to the atomic operation chain, thus avoiding the situation where business scenario experts need to perform operation orchestration and testing every time a new scenario is added, providing a basis for realizing the intelligence of scenario-driven atomic operation orchestration, reducing the expert maintenance cost, and the interaction method based on natural language description information also greatly enhances the flexibility of use and reduces the usage difficulty.

[0152] The disclosed embodiments utilize the hierarchical description and object management features unique to the communications field, standardize the design of each atomic operation, construct an atomic operation chain with a unified underlying natural language description, and design a corresponding model training process and a two-step prompt mode unique to atomic operation data based on the existing NLP model capabilities. The purpose is to use the existing large-scale pre-trained language model, make full use of various data in the communications field, and obtain a language model for atomic operation orchestration, so that the language model can automatically generate a complete atomic operation chain based on the user's scenario demand description, greatly improving the user experience while reducing the subsequent manual input.

[0153] The step division of the above various methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this disclosure. Adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the protection scope of this disclosure.

[0154] A language model training device is provided in an embodiment of the present disclosure. The specific implementation of the device can be found in the relevant description of the language model training method embodiment, which will not be repeated here. Figure 4 The schematic diagram of the structure of the device is shown, which mainly includes:

[0155] An acquisition module 401 is used to acquire a general pre-trained language model and communication scenario sample data, and pre-train the general pre-trained language model to obtain a base model;

[0156] The training module 402 is used to train the base model using the communication scenario sample data to obtain a language model for atomic operation orchestration; wherein the communication scenario sample data includes a mapping relationship between the natural language description information of the communication scenario sample and the description information of the corresponding atomic operation chain sample.

[0157] In some embodiments, the acquisition module 401 is used to: perform one-way autoregressive pre-training on the general pre-trained language model using communication field corpus to obtain the base model.

[0158] In some embodiments, the training module 402 is used to: perform first-stage prompt training on the base model using the communication scenario sample data to obtain a first-stage language model; perform second-stage prompt training on the first-stage language model using the communication scenario sample data to obtain a second-stage language model; perform instruction pattern training on the second-stage language model to obtain a language model for atomic operation orchestration.

[0159] In some embodiments, the training module 402 is used to: obtain a first type of prompt sample data; wherein the first type of prompt sample data is obtained by annotating the natural language description information of the communication scenario sample using the first type of prompt template, and the first type of prompt template is used to vertically parse the natural language description information of the communication scenario sample to prompt the relationship between the scene, phenomenon and atomic operation; use the first type of prompt sample data to train the base model to obtain a first-stage language model.

[0160] In some embodiments, the training module 402 is used to: obtain second-category prompt sample data; wherein the second-category prompt sample data is obtained by annotating the natural language description information of the communication scenario sample using the second-category prompt template, and the second-category prompt template is used to perform horizontal analysis on the natural language description information of the communication scenario sample to prompt the extension between scenarios, phenomena and atomic operations; and use the second-category prompt sample data to train the first-stage language model to obtain the second-stage language model.

[0161] In some embodiments, the training module 402 is used to: obtain natural language description information corresponding to a single data link from the natural language description information of the communication scenario sample; use the first type of prompt template to disassemble the natural language description information corresponding to the single data link from a preset dimension to obtain the first type of prompt sample data corresponding to the single data link, wherein the preset dimension includes at least one of the following: the sequential dimension of atomic operations, and the sequential dimension of scenarios and operations.

[0162] In some embodiments, the training module 402 is used to: use the natural language description information of the communication scenario samples to obtain the phenomenon corresponding to each scenario, use the phenomenon corresponding to each scenario to determine the intersection between the scenario and the phenomenon, and use the intersection between the scenario and the phenomenon to generate the second type of prompt sample data;

[0163] and / or

[0164] Using the natural language description information of the communication scenario sample to obtain each atomic operation and the business meaning corresponding to each atomic operation, using the business meaning corresponding to each of the atomic operations to determine the intersection between the atomic operations, and using the intersection between the atomic operations to generate the second type of prompt sample data;

[0165] and / or

[0166] The natural language description information of the communication scenario samples is used to obtain the phenomena corresponding to each scenario, the atomic operations corresponding to the scenario, and the business meanings corresponding to each atomic operation. The phenomena corresponding to each scenario, the atomic operations corresponding to the scenario, and the business meanings corresponding to each atomic operation are used to determine the intersections among the scenarios, phenomena, and atomic operations. The intersections among the scenarios, phenomena, and atomic operations are used to generate the second type of prompt sample data.

[0167] In some embodiments, the training module 402 is used to: obtain instruction pattern sample data; wherein the instruction pattern sample data is obtained by annotating the natural language description information of the communication scenario sample using an instruction template, and a sample data of the instruction pattern sample data includes an instruction task text and a corresponding annotated answer, and the instruction task text is used to ask questions about the relationship between scenarios, phenomena and atomic operations; use the instruction pattern sample data to perform instruction pattern training on the second-stage language model to obtain a language model for atomic operation orchestration.

[0168] An atomic operation orchestration device is provided in an embodiment of the present disclosure. The specific implementation of the device can refer to the relevant description of the atomic operation orchestration method embodiment, which will not be repeated here. Figure 5 The schematic diagram of the structure of the device is shown, which mainly includes:

[0169] An acquisition module 501 is used to acquire natural language description information of communication scenario requirements;

[0170] An analysis module 502 is used to analyze the natural language description information of the communication scenario requirement by using a language model to obtain description information of the atomic operation chain corresponding to the communication scenario requirement;

[0171] The orchestration module 503 is used to obtain at least one atomic operation constituting the atomic operation chain and perform orchestration according to the description information of the atomic operation chain.

[0172] In some embodiments, the description information of the atomic operation chain includes a description keyword of at least one atomic operation constituting the atomic operation chain, and an arrangement order of the description keyword of the at least one atomic operation;

[0173] The orchestration module 503 is used to: obtain corresponding atomic operations from an atomic operation pool according to the description keywords of the at least one atomic operation included in the atomic operation chain; wherein the atomic operation pool includes atomic operations in the communication field and names corresponding to the atomic operations, and the names include the description keywords of the corresponding atomic operations; and orchestrate the atomic operations obtained from the atomic operation pool according to the arrangement order of the description keywords of at least one atomic operation constituting the atomic operation chain.

[0174] In some embodiments, the atomic operation pool includes at least one of the following: data-type atomic operations, the names of which include at least description keywords of the data object and description keywords of the operation; task-type atomic operations, the names of which include at least description keywords of the operation and description keywords of the communication scenario.

[0175] The functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the method described in the method embodiments. The specific implementation and technical effects thereof can be referred to the description of the method embodiments above, and will not be described again here for the sake of brevity.

[0176] It should be noted that all modules involved in this embodiment are logic modules. In practical applications, a logic unit may be a physical unit, or a part of a physical unit, or may be implemented as a combination of multiple physical units. In addition, in order to highlight the innovative part of the present disclosure, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by the present disclosure, but this does not mean that there are no other units in this embodiment.

[0177] Reference Figure 6 , an embodiment of the present disclosure provides an electronic device, comprising:

[0178] at least one processor 601;

[0179] A memory 602 having at least one program stored thereon, and when the at least one program is executed by the at least one processor, the at least one processor implements the above method;

[0180] At least one I / O interface 603 is connected between the processor and the memory and is configured to implement information exchange between the processor and the memory.

[0181] Among them, the processor 601 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 602 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 603 is connected between the processor 601 and the memory 602, and can realize information interaction between the processor 601 and the memory 602, including but not limited to a data bus (Bus), etc.

[0182] In some embodiments, the processor 601 , the memory 602 , and the I / O interface 603 are connected to each other through a bus, and further connected to other components of the computing device.

[0183] This embodiment further provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method provided in this embodiment is implemented. To avoid repeated description, the specific steps of the method are not repeated here.

[0184] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods applied for above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0185] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0186] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.

[0187] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A language model training method, characterized in that: include: Obtaining a general pre-trained language model and communication scenario sample data, and pre-training the general pre-trained language model to obtain a base model; The base model is trained using the communication scenario sample data to obtain a language model for atomic operation orchestration; wherein the communication scenario sample data includes a mapping relationship between natural language description information of the communication scenario sample and description information of the corresponding atomic operation chain sample.

2. The method according to claim 1, characterized in that: The pre-training of the general pre-trained language model to obtain a base model includes: The universal pre-trained language model is subjected to one-way autoregressive pre-training using communication domain corpus to obtain the base model.

3. The method according to claim 1, characterized in that The base model is trained using the communication scenario sample data to obtain a language model for atomic operation arrangement, including: Using the communication scenario sample data, the base model is trained with the first stage prompt to obtain a first stage language model; Using the communication scenario sample data, the first-stage language model is trained with second-stage prompts to obtain a second-stage language model; The second-stage language model is trained with instruction patterns to obtain a language model for atomic operation orchestration.

4. The method according to claim 3, characterized in that The base model is trained with the communication scenario sample data for the first stage prompt to obtain a first stage language model, including: Acquire first-category prompt sample data; wherein the first-category prompt sample data is obtained by annotating natural language description information of communication scenario samples using a first-category prompt template, and the first-category prompt template is used to vertically parse the natural language description information of the communication scenario samples to prompt the association between scenarios, phenomena, and atomic operations; The base model is trained using the first type of prompt sample data to obtain a first-stage language model.

5. The method according to claim 4, characterized in that The using the communication scenario sample data to perform second-stage prompt training on the first-stage language model to obtain the second-stage language model includes: Acquire the second type of prompt sample data; wherein the second type of prompt sample data is obtained by annotating the natural language description information of the communication scenario sample using the second type of prompt template, and the second type of prompt template is used to perform horizontal analysis on the natural language description information of the communication scenario sample to prompt the extension between the scenario, phenomenon and atomic operation; The first-stage language model is trained using the second-category prompt sample data to obtain a second-stage language model.

6. The method according to claim 4, characterized in that The obtaining of the first type of prompt sample data includes: Obtaining natural language description information corresponding to a single data link from natural language description information of the communication scenario sample; The first type of prompt template is used to disassemble the natural language description information corresponding to the single data chain from a preset dimension to obtain the first type of prompt sample data corresponding to the single data chain, wherein the preset dimension includes at least one of the following: a sequential dimension of atomic operations, and a sequential dimension of scenes and operations.

7. The method according to claim 5, characterized in that The obtaining of the second type of prompt sample data includes: Using the natural language description information of the communication scenario samples, obtaining the phenomenon corresponding to each scenario, using the phenomenon corresponding to each scenario, determining the intersection between the scenario and the phenomenon, and using the intersection between the scenario and the phenomenon to generate the second type of prompt sample data; and / or Using the natural language description information of the communication scenario sample to obtain each atomic operation and the business meaning corresponding to each atomic operation, using the business meaning corresponding to each of the atomic operations to determine the intersection between the atomic operations, and using the intersection between the atomic operations to generate the second type of prompt sample data; and / or The natural language description information of the communication scenario samples is used to obtain the phenomena corresponding to each scenario, the atomic operations corresponding to the scenario, and the business meanings corresponding to each atomic operation. The phenomena corresponding to each scenario, the atomic operations corresponding to the scenario, and the business meanings corresponding to each atomic operation are used to determine the intersections among the scenarios, phenomena, and atomic operations. The intersections among the scenarios, phenomena, and atomic operations are used to generate the second type of prompt sample data.

8. The method according to claim 3, characterized in that The performing instruction pattern training on the second-stage language model to obtain a language model for atomic operation arrangement includes: Acquire instruction pattern sample data; wherein the instruction pattern sample data is obtained by annotating natural language description information of a communication scenario sample using an instruction template, and one sample data of the instruction pattern sample data includes an instruction task text and a corresponding annotated answer, and the instruction task text is used to ask questions about the relationship between scenarios, phenomena and atomic operations; The instruction pattern sample data is used to perform instruction pattern training on the second-stage language model to obtain a language model for atomic operation arrangement.

9. An atomic operation arrangement method, characterized in that: include: Obtain natural language description information of communication scenario requirements; A language model is used to analyze the natural language description information of the communication scenario requirements to obtain description information of the atomic operation chain corresponding to the communication scenario requirements; wherein the language model is trained using the method described in any one of claims 1 to 7; According to the description information of the atomic operation chain, at least one atomic operation constituting the atomic operation chain is obtained and arranged.

10. The method according to claim 9, characterized in that The description information of the atomic operation chain includes a description keyword of at least one atomic operation constituting the atomic operation chain, and an arrangement order of the description keyword of the at least one atomic operation; The acquiring and arranging at least one atomic operation constituting the atomic operation chain according to the description information of the atomic operation chain includes: According to the description keywords of the at least one atomic operation included in the atomic operation chain, corresponding atomic operations are obtained from an atomic operation pool; wherein the atomic operation pool includes atomic operations in the communication field and names corresponding to the atomic operations, and the names include description keywords of the corresponding atomic operations; The atomic operations obtained from the atomic operation pool are arranged according to the arrangement order of the description keywords of at least one atomic operation constituting the atomic operation chain.

11. The method according to claim 10, characterized in that The atomic operation pool includes at least one of the following: A data-based atomic operation, wherein the name of the data-based atomic operation includes at least a description keyword of the data object and a description keyword of the operation; The task-type atomic operation, wherein the name of the task-type atomic operation at least includes a description keyword of the operation and a description keyword of the communication scenario.

12. A language model training device, characterized in that: include: An acquisition module, used to acquire a general pre-trained language model and communication scenario sample data, and pre-train the general pre-trained language model to obtain a base model; A training module is used to train the base model using the communication scenario sample data to obtain a language model for atomic operation orchestration; wherein the communication scenario sample data includes a mapping relationship between the natural language description information of the communication scenario sample and the description information of the corresponding atomic operation chain sample.

13. An atomic operation arrangement device, characterized in that: include: An acquisition module, used to acquire natural language description information of communication scenario requirements; An analysis module, used to analyze the natural language description information of the communication scenario requirements using a language model to obtain description information of the atomic operation chain corresponding to the communication scenario requirements; wherein the language model is trained using the method described in any one of claims 1 to 7; The orchestration module is used to obtain and orchestrate at least one atomic operation constituting the atomic operation chain according to the description information of the atomic operation chain.

14. An electronic device, characterized in that: include: at least one processor; A memory having at least one program stored thereon, wherein when the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 8 or implements the method according to any one of claims 9 to 11; At least one I / O interface is connected between the processor and the memory and is configured to implement information interaction between the processor and the memory.

15. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 8 or the method according to any one of claims 9 to 11 is implemented.

Images