Method, device and equipment for generating large network cutover model

Abstract

The invention provides a method, device and equipment for generating a large network cutover model, and the method comprises the steps: collecting cutover corpora, carrying out the similarity corpus deduplication and semantic complementation cleaning of the collected corpora, and storing the collected corpora as a document in a word or pdf format; the method comprises the following steps of: labeling a stored document in a mode of combining manual labeling and intelligent labeling, and outputting a labeled sample in a json format according to a labeling result which is a question-answer pair of input and output; taking the labeled sample as a professional field training sample, extracting data from the open source labeled sample as a general sample, and mixing the professional field training sample and the general sample as a final training sample; an open source large model is used as a pre-training model, a final training sample is used to carry out SFT fine tuning through a lora method, a cutover large model is generated, and the generated cutover large model is evaluated by constructing a cutover knowledge question library and a cutover large model answer score.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of network processing technology, and more particularly to a method, device and apparatus for generating a large network cutover model. Background Art

[0002] In the communication network operation and maintenance scenario, network cutover is a daily key task. Cutover involves various types of network element operations and requires deep accumulation. The cutover process is complicated. The current cutover script generation mainly relies on manual cutover instructions. Operators need to be very proficient in the instructions of different equipment manufacturers and models, relying entirely on expert experience. However, due to the large number of manufacturers and models and the complexity of the business, it is very difficult to write cutover scripts.

[0003] In addition, most of the large models that have been opened are general large models, which do not have a deep understanding of the cutover business and have poor network cutover instruction capabilities, resulting in the written instructions not being able to be actually used, making it difficult to decompose and automatically execute cutover requirements.

[0004] Therefore, we need to generate a large model of the network cutover scenario for the network cutover business, improve the large model's own understanding of cutover knowledge, and provide a powerful large model foundation for subsequent cutover robots to automatically implement the end-to-end cutover process. Summary of the invention

[0005] To solve the above problems, the present invention realizes the functions of automatic generation of cutover scripts, understanding of cutover requirements, verification of cutover configuration instructions, etc. through big model question and answer, and provides an end-to-end cutover big model generation solution, which can be used as the basis for the automatic execution process of cutover and reduce human participation; it uses corpus cleaning, information reorganization, manual annotation based on business understanding and intelligent annotation solution based on document understanding, which takes into account both business practicality and the comprehensiveness of corpus and the efficiency of annotation work, and provides a specific and operational process for the rapid generation of cutover big models.

[0006] According to an embodiment of the present invention, a method, apparatus and device for generating a large network cutover model are provided.

[0007] In a first aspect of the present invention, a method for generating a large network cutover model is provided. The method comprises:

[0008] S01: Collect the cutover corpus, remove duplicates and complete the semantics of the collected corpus, and then store it in a document in word or pdf format;

[0009] S02: annotate the stored documents by combining manual annotation and intelligent annotation. The annotation results are question-answer pairs of input and output. The annotated samples are output in JSON format.

[0010] S03: Use the labeled samples as professional training samples, extract data from open source labeled samples as general samples, and mix the professional training samples and general samples as the final training samples;

[0011] S04: Use the open source big model as the pre-training model, use the final training sample to perform SFT fine-tuning through the lora method to generate a cutover big model, and then evaluate the cutover big model generated this time by constructing a cutover knowledge question library and scoring the cutover big model answers.

[0012] Furthermore, the cutover corpus described in S01 includes: a cutover work order, a cutover plan, a cutover script, a device product manual, a configuration specification, and a network planning architecture.

[0013] Furthermore, the specific steps of deduplicating the similar corpus described in S01 are: dividing the corpus into blocks according to paragraphs and word counts, and storing the divided corpus in a word vector library, forming a comparison corpus with every two spliced ​​corpora, and using a cosine similarity algorithm to compare the similarity of the word vector blocks in the corpus. If the similarity exceeds 80%, the larger file is retained and the smaller file is deleted. The similarities of all spliced ​​corpora are compared in turn to deduplicate the entire corpus.

[0014] Furthermore, the specific steps of the semantic completion described in S01 are:

[0015] Read the deduplicated corpus line by line and extract the image information, table information, code information, and text information;

[0016] Use the OCR recognition interface to extract keywords from the image information, and use the semantics of the big model to understand the OCR recognition content. Use the OCR recognition content and the 500 words above and below the image in the document as the question and answer content, ask the big model, understand the OCR content, regenerate the corresponding content, fill in the missing key information of the image semantics, and save the key information to the corresponding location;

[0017] Read table information, read the information in rows or columns, use the big model to understand and reorganize the information, and save the reorganized information to the corresponding location;

[0018] Read the script information and use the separator to identify the script information here.

[0019] Furthermore, the manual labeling described in S02 is used to label the collected demand work orders, cutover plans and cutover script data, which is divided into cutover demand type labeling, cutover demand key information labeling and cutover script labeling; the cutover demand type labeling uses the demand work order data to label the type of cutover work order according to the content of the cutover demand order; the cutover demand key information labeling labels the key information required for this type according to the cutover type; the cutover script labeling uses the cutover plan and cutover script data to label the cutover generation script according to the cutover type and key information, and provides the corresponding cutover script, fallback script, and verification information required for the completion of this type of cutover.

[0020] Furthermore, S02 is used to annotate the collected introductory documents, specification documents, and operation and maintenance manuals, and the specific steps are: split the document according to the small chapter directory, annotate the script part for the corpus collection and cleaning module, and the large model explains the questions according to the main function of this script and the meaning of each command block in the script; for the annotation of other text information, it is split into term explanation, main function summary, process introduction, and exception handling method, which matches the professional knowledge required in the cutover process.

[0021] Furthermore, the data of 10% to 50% of the domain samples are sampled from the open source annotated samples as described in S03 as the general samples.

[0022] Furthermore, the specific steps of evaluating the generated cutover big model by constructing a cutover knowledge question library and answering and scoring the cutover big model as described in S04 are as follows: construct N questions and standard answers from five aspects: cutover script generation, cutover requirement understanding, cutover problem handling, networking specifications, and networking structure; answer 5*N questions with the fine-tuned network cutover big model and give specific answers; use word2vec similarity to compare the similarity between the correct answer and the standard answer; and take the result of adding the similarity of each question * 5*N as the final score.

[0023] In a second aspect of the present invention, a device for generating a large network cutover model is provided. The device comprises:

[0024] Corpus collection module: used to collect the cutover corpus, remove similar corpus duplication and complete the semantics of the collected corpus, and then store it in a document in word or pdf format;

[0025] Corpus annotation module: used to annotate stored documents by combining manual annotation and intelligent annotation. The annotation results are input and output question-answer pairs, and the annotated samples are output in JSON format.

[0026] Sample formation module: used to use the labeled samples as professional field training samples, extract data from open source labeled samples as general samples, and mix the professional field training samples and general samples as the final training samples;

[0027] Model generation module: used to use the open source big model as the pre-training model, use the final training sample to perform SFT fine-tuning through the lora method, generate the cutover big model, and then evaluate the cutover big model generated this time by constructing a cutover knowledge question library and scoring the cutover big model answers.

[0028] In a third aspect of the present invention, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method according to the first aspect of the present invention is implemented.

[0029] In a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present invention is implemented.

[0030] The meanings of the above mentioned English abbreviations:

[0031] input: input

[0032] output: output

[0033] json: JavaScript Object Notation, a lightweight data exchange format

[0034] lora: Learnable Re-Weighting, a learnable reweighting model for neural networks

[0035] SFT: Supervised Fine-Tun-ing, supervised fine-tuning

[0036] OCR: Optical Character Recognition, optical character recognition

[0037] word2vec: A group of related models used to generate word vectors

[0038] The present invention realizes functions such as automatic generation of cutover scripts, understanding of cutover requirements, and verification of cutover configuration instructions through big model question and answer, and provides an end-to-end cutover big model generation solution, which can serve as the basis for the automatic execution process of cutover and reduce human participation; the present invention uses corpus cleaning, information reorganization, manual annotation based on business understanding, and intelligent annotation solutions based on document understanding, which take into account both business practicality and the comprehensiveness of corpus and the efficiency of annotation work, and provides a specific and operable process for quickly generating a cutover big model.

[0039] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings.

[0041] Figure 1 A flow chart of a method for generating a large network cutover model according to an embodiment of the present invention is shown;

[0042] Figure 2 A block diagram of a device for generating a large network cutover model according to an embodiment of the present invention is shown;

[0043] Figure 3 A schematic diagram of a device for generating a large network cutover model according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] According to the implementation mode of the present invention, a method, device and equipment for generating a large model of network cutover are proposed. Through the large model question and answer, the functions of automatic generation of cutover scripts, understanding of cutover requirements, and verification of cutover configuration instructions are realized, and an end-to-end cutover large model generation solution is provided, which can be used as the basis for the automatic execution process of cutover and reduce human participation. The use of corpus cleaning, information reorganization, manual annotation based on business understanding and intelligent annotation solutions based on document understanding takes into account both business practicality and the comprehensiveness of the corpus and the efficiency of annotation work, providing a specific and operational process for the rapid generation of a large cutover model.

[0046] The principle and spirit of the present invention are explained in detail below with reference to several representative embodiments of the present invention.

[0047] Figure 1 1 is a flow chart of a method for generating a large network cutover model according to an embodiment of the present invention. The method comprises:

[0048] S01: Collect the cutover corpus, remove duplicates and complete the semantics of the collected corpus, and then store it in a document in word or pdf format;

[0049] S02: annotate the stored documents by combining manual annotation and intelligent annotation. The annotation results are question-answer pairs of input and output. The annotated samples are output in JSON format.

[0050] S03: Use the labeled samples as professional training samples, extract data from open source labeled samples as general samples, and mix the professional training samples and general samples as the final training samples;

[0051] S04: Use the open source big model as the pre-training model, use the final training sample to perform SFT fine-tuning through the lora method to generate a cutover big model, and then evaluate the cutover big model generated this time by constructing a cutover knowledge question library and scoring the cutover big model answers.

[0052] It should be noted that, although the operations of the method of the present invention are described in a specific order in the above embodiments and the accompanying drawings, this does not require or imply that the operations must be performed in the specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0053] In order to explain the above method of generating a large network cutover model more clearly, a specific embodiment is described below. However, it should be noted that the embodiment is only for better illustrating the present invention and does not constitute an improper limitation to the present invention.

[0054] The following is a specific example to further illustrate the method of generating a large network cutover model in more detail:

[0055] Collect the following types of corpora, with each type of document no less than 500M:

[0056] Work order requirement corpus: historical cutover work orders, including original requirement orders and subsequent cross-departmental collaborative processing orders.

[0057] Cutover plan corpus: verified successful cutover plans, including: cutover theme and content, cutover execution process, cutover script, verification information collection instructions, rollback script, and exception handling plan.

[0058] Network element information corpus: contains various network element information: device name, device model, port, link, manufacturer and other information, including the product manual corresponding to the network element.

[0059] Network architecture and specification corpus: Contains the networking architecture of general networks, networking specification documents, white papers, technical documents, maintenance manuals, etc.

[0060] The corpus is divided into blocks according to paragraphs and word counts: within 1000 words, and the divided corpus is stored in the word vector library. The similarity of each block is compared between two files, and the similarity of the word vector blocks in the document is compared using the cosine similarity algorithm. If the similarity exceeds 80%, the larger file is retained and the smaller file is deleted. According to this method, the similarity of all blocks of all documents is compared in turn to remove duplicates from the entire corpus.

[0061] Read the deduplicated document line by line to extract the image information, table information, code information, and text information.

[0062] Use the OCR recognition interface to extract keywords from the image information, and use the semantics of the big model to understand the OCR recognition content. Use the OCR recognition content and the 500 words above and below the image in the document as question and answer content, ask questions to the big model, understand the OCR content, regenerate the corresponding content, fill in the missing key image semantic information, and save the key information to the corresponding location.

[0063] Read table information, read the information in rows or columns, use the big model to understand and reorganize the information, and save the reorganized information to the corresponding location.

[0064] Read the script information and use the separator to identify the script information here.

[0065] Save the files processed in the above steps into pdf format.

[0066] The spliced ​​corpus is annotated by combining manual annotation and intelligent annotation. The annotation results are unified into input and output question-answer pairs, and the final results are output in JSON format.

[0067] Manual labeling method: mainly manually labeling the collected demand work orders, cutover plans, and cutover script data.

[0068] Cutover requirement type labeling: Use the requirement work order data and label the type of the cutover work order according to the content of the cutover requirement order. For example, the cutover requirement content is "metropolitan area network expansion 100G". The labeling result is: {"input": "Judge the cutover type of the content: metropolitan area network expansion 100G", "output": "Link adjustment, expansion"}.

[0069] Cutover requirement key information annotation: According to the cutover type, mark the key information required for this type. {"input":"Key information for judging the cutover type: link adjustment-capacity expansion","output":"A-side port, A-side device, B-side port, B-side device, capacity expansion bandwidth"}.

[0070] Cutover script annotation: Use the cutover plan and cutover script data to annotate the cutover script according to the cutover type and key information. Provide the corresponding cutover script, fallback script, and verification information required for this type of cutover to complete.

[0071] The input of this type of annotation is the annotation task description (generate cutover script, generate rollback script, generate verification information, etc.), cutover type and key information, and the output is the cutover script, rollback script, and verification information required for this type of cutover. For example: {"input: Generate the corresponding cutover script based on the following information: the cutover type is link adjustment-capacity expansion, the key information is: A-side port: eth-1 / 2 / 3, A-side device: dev-01, B-side port: eth-5 / 1 / 2, B-side device: dev-02, expansion bandwidth: 100G","output":"display xxx"}

[0072] Intelligent annotation method: mainly annotate the collected introductory documents, specification documents, and operation and maintenance manuals. First, split the documents. Then use the question-and-answer function of the generative large model to generate annotations.

[0073] This type of document is well organized and is divided into small chapters.

[0074] For the part of the script marked by the corpus collection and cleaning module, the big model asks questions according to two types of tasks: one is the main function of this script, and the other is the explanation of the meaning of each command block in the script.

[0075] The annotations of other text information are divided into several categories: explanation of terms, summary of main functions, process introduction, and exception handling methods, which match the professional knowledge required in the cutover process.

[0076] For example: The prompt words for the big model question of explaining terms: Please extract the key names from the content of [] and explain the names in it. [The metropolitan area network is the networking structure of the city, and the main network elements are BARS, SR, OLT, etc.]”

[0077] Corresponding results: "input":"What is a metropolitan area network", "output":"The metropolitan area network is the networking structure of cities, and the main network elements include BARS, SR, OLT, etc."

[0078] Prompt words for the question of the function summary model: Please summarize the main content from [ ] content, and ask 5 questions for each content. [The metropolitan area network is the networking structure of the city, and the main network elements are BARS, SR, OLT, etc. ]”

[0079] The corresponding result: "input":"The main content in [] is","output":"the concept of metropolitan area network".

[0080] The above labeled samples are used as professional field training samples, and 10%, 25%, or 50% of the field samples are sampled from the open source labeled samples as general samples, and the two samples are mixed as the final training samples. The specific ratio can be reset according to the evaluation results later.

[0081] Use the open source large model as the pre-training model, use the above-constructed training samples to perform SFT fine-tuning through the lora method, and finally use the fine-tuned large model as the network cutover large model.

[0082] Construct network cutover assessment use cases and construct samples through questions and answers. Mainly from cutover script generation, cutover requirement understanding, cutover problem handling, networking specifications, and networking structure questions, construct 20 questions and standard answers for each aspect.

[0083] Use the fine-tuned network to cut the large model to answer 100 questions and give specific answers. Use word2vec similarity to compare the similarity between the correct answer and the standard answer, and add the result of the similarity of each question * 100 as the final score.

[0084] Based on the same inventive concept, the present invention also proposes a device for generating a large network cutover model. The implementation of the device can refer to the implementation of the above method, and the repeated parts will not be repeated. Figure 2 As shown, the device 100 includes:

[0085] Corpus collection module 101: used to collect the cutover corpus, and perform similarity corpus deduplication and semantic completion cleaning on the collected corpus, and then store it in a document in word or pdf format;

[0086] Corpus annotation module 102: used to annotate the stored documents by combining manual annotation and intelligent annotation. The annotation result is a question-answer pair of input and output, and the annotated samples are output in JSON format;

[0087] Sample forming module 103: used to use the labeled samples as professional field training samples, extract data from open source labeled samples as general samples, and mix the professional field training samples and general samples as final training samples;

[0088] Model generation module 104: used to use the open source big model as the pre-training model, use the final training sample to perform SFT fine-tuning through the lora method, generate the cutover big model, and then evaluate the cutover big model generated this time by constructing a cutover knowledge question library and the cutover big model answer score.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0090] like Figure 3 As shown, the device includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The CPU, ROM and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0091] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0092] The processing unit performs the various methods and processes described above, such as methods S01 to S04. For example, in some embodiments, methods S01 to S04 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S01 to S04 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to perform methods S01 to S04 in any other appropriate manner (e.g., by means of firmware).

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0094] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present invention. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0097] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for generating a large network cutover model, characterized in that: The method includes: S01: Collect the cutover corpus, remove duplicates and complete the semantics of the collected corpus, and then store it in a document in word or pdf format; S02: annotate the stored documents by combining manual annotation and intelligent annotation. The annotation results are question-answer pairs of input and output. The annotated samples are output in JSON format. S03: Use the labeled samples as professional training samples, extract data from open source labeled samples as general samples, and mix the professional training samples and general samples as the final training samples; S04: Use the open source big model as the pre-training model, use the final training sample to perform SFT fine-tuning through the lora method to generate a cutover big model, and then evaluate the cutover big model generated this time by constructing a cutover knowledge question library and scoring the cutover big model answers.

2. The method for generating a large network cutover model according to claim 1, characterized in that: The cutover corpus described in S01 includes: a cutover work order, a cutover plan, a cutover script, a device product manual, a configuration specification, and a network planning architecture.

3. The method for generating a large network cutover model according to claim 1, characterized in that: The specific steps of deduplication of similar corpora described in S01 are: divide the corpus into blocks according to paragraphs and word counts, and store the divided corpora in a word vector library, form a comparison corpus with every two spliced ​​corpora, and use the cosine similarity algorithm to compare the similarity of the word vector blocks in the corpus. If the similarity exceeds 80%, retain the larger file and delete the smaller file, and compare the similarities of all spliced ​​corpora in turn to deduplicate the entire corpus.

4. The method for generating a large network cutover model according to claim 1, characterized in that: The specific steps of semantic completion described in S01 are: Read the deduplicated corpus line by line and extract the image information, table information, code information, and text information; Use the OCR recognition interface to extract keywords from the image information, and use the semantics of the big model to understand the OCR recognition content. Use the OCR recognition content and the 500 words above and below the image in the document as the question and answer content, ask the big model, understand the OCR content, regenerate the corresponding content, fill in the missing key information of the image semantics, and save the key information to the corresponding location; Read table information, read the information in rows or columns, use the big model to understand and reorganize the information, and save the reorganized information to the corresponding location; Read the script information and use the separator to identify the script information here.

5. The method for generating a large network cutover model according to claim 1, characterized in that: The manual labeling described in S02 is used to label the collected demand work orders, cutover plans and cutover script data, which is divided into cutover demand type labeling, cutover demand key information labeling and cutover script labeling; the cutover demand type labeling uses the demand work order data to label the type of cutover work order according to the content of the cutover demand order; the cutover demand key information labeling uses the cutover type to label the key information required for this type; the cutover script labeling uses the cutover plan and cutover script data to label the cutover generation script according to the cutover type and key information, and provides the corresponding cutover script, fallback script, and verification information required for the completion of this type of cutover.

6. The method for generating a large network cutover model according to claim 1, characterized in that: S02 is used to annotate the collected introductory documents, specification documents, and operation and maintenance manuals. The specific steps are: split the documents according to the small chapter directory, annotate the script part of the corpus collection and cleaning module, and the large model explains the questions according to the main function of this script and the meaning of each command block in the script; for the annotation of other text information, split it into term explanation, main function summary, process introduction, and exception handling method to match the professional knowledge required in the cutover process.

7. The method for generating a large network cutover model according to claim 1, characterized in that: As described in S03, 10% to 50% of the domain samples are sampled from the open source annotated samples as the general samples.

8. The method for generating a large network cutover model according to claim 1, characterized in that: The specific steps of evaluating the generated cutover big model by constructing a cutover knowledge question library and answering and scoring the cutover big model as described in S04 are as follows: construct N questions and standard answers from five aspects: cutover script generation, cutover requirement understanding, cutover problem handling, networking specifications, and networking structure; answer 5*N questions with the fine-tuned network cutover big model and give specific answers; use word2vec similarity to compare the similarity between the correct answer and the standard answer; and add the similarity of each question * 5*N as the final score.

9. A device for generating a large network cutover model, characterized in that: The device includes: Corpus collection module: used to collect the cutover corpus, remove similar corpus duplication and complete the semantics of the collected corpus, and then store it in a document in word or pdf format; Corpus annotation module: used to annotate stored documents by combining manual annotation and intelligent annotation. The annotation results are input and output question-answer pairs, and the annotated samples are output in JSON format. Sample formation module: used to use the labeled samples as professional field training samples, extract data from open source labeled samples as general samples, and mix the professional field training samples and general samples as the final training samples; Model generation module: used to use the open source big model as the pre-training model, use the final training sample to perform SFT fine-tuning through the lora method, generate the cutover big model, and then evaluate the cutover big model generated this time by constructing a cutover knowledge question library and scoring the cutover big model answers.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images