Data processing method and device based on code large language model

Abstract

The present disclosure provides a data processing method and device based on a code large language model. The method includes: obtaining the data to be processed and the target task corresponding to the data to be processed; using the large language model to extract the prompt guidance information of the data to be processed according to the target task; converting the data to be processed and the prompt guidance information into a data code and a prompt code in a preset format respectively; obtaining the target code large language model corresponding to the target task from the model library; using the target code large language model to process the data code according to the prompt code to obtain the processing result. The above technical means are used to solve the problem of low information extraction efficiency and accuracy in the prior art.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data processing method and device based on a large code language model. Background Art

[0002] In recent years, large language models, such as GPT, have become core technologies in the field of NLP. They have demonstrated excellent performance on multiple NLP tasks through pre-training and fine-tuning. These models usually use deep learning technology and are pre-trained on large-scale corpora to capture the rich characteristics and complex structures of language. Especially in the field of information extraction, these models have demonstrated sensitivity to subtle language features and the ability to understand complex contexts.

[0003] Despite this, current information extraction systems still face many challenges. For example, models often have difficulty fully capturing and understanding the multi-level and multi-dimensional information in texts, resulting in low accuracy in information extraction. In addition, traditional information extraction systems are inefficient when processing large-scale, real-time updated data streams, making it difficult to meet the needs of real-time information processing. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a data processing method, device, electronic device and computer-readable storage medium based on a large code language model to solve the problems of low information extraction efficiency and accuracy in the prior art.

[0005] In a first aspect of an embodiment of the present disclosure, a data processing method based on a code large language model is provided, including: obtaining data to be processed and a target task corresponding to the data to be processed; using a large language model to extract prompt guidance information of the data to be processed according to the target task; converting the data to be processed and the prompt guidance information into data code and prompt code in a preset format respectively; obtaining a target code large language model corresponding to the target task from a model library; and using the target code large language model to process the data code according to the prompt code to obtain a processing result.

[0006] According to a second aspect of the disclosed embodiments, a data processing device based on a code large language model is provided, including: a first acquisition module, configured to acquire data to be processed and a target task corresponding to the data to be processed; an extraction module, configured to use the large language model to extract prompt guidance information of the data to be processed according to the target task; a conversion module, configured to convert the data to be processed and the prompt guidance information into data codes and prompt codes in preset formats, respectively; a second acquisition module, configured to acquire a target code large language model corresponding to the target task from a model library; and a processing module, configured to use the target code large language model to process the data code according to the prompt code to obtain a processing result.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0009] Compared with the prior art, the disclosed embodiment has the following beneficial effects: obtaining the data to be processed and the target task corresponding to the data to be processed; using the large language model to extract the prompt guidance information of the data to be processed according to the target task; converting the data to be processed and the prompt guidance information into data code and prompt code in a preset format respectively; obtaining the target code large language model corresponding to the target task from the model library; using the target code large language model to process the data code according to the prompt code to obtain the processing result. The above technical means can solve the problem of low efficiency and accuracy of information extraction in the prior art, thereby improving the efficiency and accuracy of information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 It is a flowchart of a data processing method based on a code language model provided by an embodiment of the present disclosure;

[0012] Figure 2 It is a flowchart of a method for training a large code language model provided by an embodiment of the present disclosure;

[0013] Figure 3 It is a schematic diagram of the structure of a data processing device based on a code large language model provided by an embodiment of the present disclosure;

[0014] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details that hinder the description of the present disclosure.

[0016] A data processing method and device based on a large code language model according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0017] Figure 1 It is a flowchart of a data processing method based on a large code language model provided by an embodiment of the present disclosure. Figure 1 The data processing method based on the code language model can be executed by a computer or a server, or by software on the computer or the server. Figure 1 As shown, the data processing method based on the code large language model includes:

[0018] S101, obtaining data to be processed and target tasks corresponding to the data to be processed;

[0019] S102, using the large language model to extract prompt guidance information of the data to be processed according to the target task;

[0020] S103, converting the data to be processed and the prompt guidance information into a data code and a prompt code in a preset format respectively;

[0021] S104, obtaining a target code language model corresponding to the target task from a model library;

[0022] S105, using the target code large language model to process the data code according to the prompt code to obtain a processing result.

[0023] The target task can be a named entity recognition task, a named entity relationship extraction task, an event extraction task, etc. The named entity recognition task is to identify specific entities in the text; the named entity relationship extraction task is to identify the semantic relationship between entities; the event extraction task is to identify events and their related elements in the text (such as event participants, time, and location). The data to be processed is the data that needs to be processed for the target task.

[0024] Large Language Model (LLM) is an artificial intelligence model trained with a large corpus that can understand and generate natural language text. Large language models can be BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), etc.

[0025] The model library stores code big language models corresponding to multiple tasks, and the target code big language model corresponding to the target task is one of them. The code big language model and the big language model can be the same model, the difference is that the big language model understands and generates natural language text, and the code big language model understands and generates the preset format code corresponding to the natural language text.

[0026] According to the technical solution provided by the embodiment of the present application, the data to be processed and the target task corresponding to the data to be processed are obtained; the prompt guidance information of the data to be processed is extracted according to the target task using the large language model; the data to be processed and the prompt guidance information are respectively converted into data code and prompt code in a preset format; the target code large language model corresponding to the target task is obtained from the model library; the data code is processed according to the prompt code using the target code large language model to obtain the processing result. The above technical means can solve the problem of low efficiency and accuracy of information extraction in the prior art, thereby improving the efficiency and accuracy of information extraction.

[0027] Before using the large language model to extract prompt guidance information of the data to be processed according to the target task, the method also includes: using the large language model to extract training prompt guidance information of the training data corresponding to the target task according to the target task; calculating the loss between the training prompt guidance information and the label of the training data, optimizing the model parameters of the large language model based on the loss, and completing the training of the large language model.

[0028] The labels of the training data are the prompt guidance information annotated in advance. The trained large language model can extract the prompt guidance information of the to-be-processed data corresponding to the target task. When the target task is a named entity recognition task, the prompt guidance information is the limiting information of the entity to be extracted, such as the type of entity, such as a place or a product. When the target task is a named entity relationship extraction task, the prompt guidance information is the limiting information of the semantic relationship between the entities to be extracted, such as the semantic relationship is a belonging relationship, a causal relationship, a composition relationship, and a conditional relationship. When the target task is an event extraction task, the prompt guidance information is the limiting information of the event to be extracted, such as the event participants, time, and location.

[0029] Belonging: one entity is part of another entity; Cause-Effect: one event or entity is the cause of another event or result; Composition: one entity is composed of multiple sub-entities; Condition: one entity is the condition for the occurrence of another entity.

[0030] Furthermore, before obtaining the target code language model corresponding to the target task from the model library, the method also includes: obtaining training data corresponding to various tasks; training the code language model using the training data corresponding to various tasks; fine-tuning the code language model trained with the training data corresponding to various tasks by the Low-Rank Adaptation method to obtain the code language model corresponding to various tasks; and saving the code language model corresponding to various tasks into the model library.

[0031] Use the training data corresponding to each task to train the code language model, and you will get the code language model corresponding to the task. Then use the Low-Rank Adaptation method to fine-tune the code language model corresponding to the task, and finally get the code language model corresponding to the task. The Low-Rank Adaptation method fine-tunes the model by building lora branches in certain network layers of the model, and improving the performance of the model by optimizing the lora branches.

[0032] Figure 2 FIG. 1 is a flow chart of a method for training a large code language model provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0033] For various tasks:

[0034] S201, using a large language model to extract training prompt guidance information of training data corresponding to the task according to the task;

[0035] S202, converting the training data and the training prompt guidance information into a training data code and a training prompt code in a preset format respectively;

[0036] S203, using the code language model to process the training data code according to the training prompt code to obtain a training processing result;

[0037] S204, calculating the loss between the training processing result and the label of the training data, optimizing the model parameters of the code large language model according to the loss, and obtaining the code large language model corresponding to the task.

[0038] In some embodiments, first training data corresponding to a named entity recognition task is obtained; a large code language model is trained using the first training data; the large code language model trained with the first training data is fine-tuned using a Low-Rank Adaptation method to obtain a first large code language model; and the first large code language model is saved in a model library.

[0039] The training data corresponding to the named entity recognition task is recorded as the first training data, and the code large language model is trained using the first training data: the training prompt guidance information of the first training data is extracted using the large language model according to the named entity recognition task; the training data and the training prompt guidance information are respectively converted into training data codes and training prompt codes in preset formats; the code large language model is used to process the training data code according to the training prompt code to obtain the training processing result; the loss between the training processing result and the label of the first training data is calculated, and the model parameters of the code large language model are optimized according to the loss to obtain the trained code large language model. The label of the first training data is the result of processing the first training data according to the named entity recognition task, which is marked in advance for the first training data.

[0040] The code language model trained with the first training data is fine-tuned by the Low-Rank Adaptation method to obtain a first code language model, which is a code language model corresponding to the named entity recognition task. The first code language model is saved in the model library.

[0041] In some embodiments, second training data corresponding to the named entity relationship extraction task is obtained; a large code language model is trained using the second training data; the large code language model trained with the second training data is fine-tuned by a Low-Rank Adaptation method to obtain a second large code language model; and the second large code language model is saved in a model library.

[0042] The training data corresponding to the named entity relationship extraction task is recorded as the second training data, and the second training data is used to train the code large language model: the training prompt guidance information of the second training data is extracted by the large language model according to the named entity relationship extraction task; the training data and the training prompt guidance information are respectively converted into the training data code and the training prompt code in a preset format; the code large language model is used to process the training data code according to the training prompt code to obtain the training processing result; the loss between the training processing result and the label of the second training data is calculated, and the model parameters of the code large language model are optimized according to the loss to obtain the trained code large language model. The label of the second training data is the result of processing the second training data according to the named entity relationship extraction task, which is marked in advance for the second training data.

[0043] The code language model trained with the second training data is fine-tuned by the Low-Rank Adaptation method to obtain a second code language model, which is a code language model corresponding to the named entity relationship extraction task. The second code language model is saved in the model library.

[0044] In some embodiments, third training data corresponding to the event extraction task is obtained; a large code language model is trained using the third training data; the large code language model trained with the third training data is fine-tuned by a Low-Rank Adaptation method to obtain a third large code language model; and the third large code language model is saved in a model library.

[0045] The training data corresponding to the event extraction task is recorded as the third training data, and the third training data is used to train the code large language model: the training prompt guidance information of the third training data is extracted by the large language model according to the event extraction task; the training data and the training prompt guidance information are respectively converted into training data codes and training prompt codes in preset formats; the code large language model is used to process the training data code according to the training prompt code to obtain the training processing result; the loss between the training processing result and the label of the third training data is calculated, and the model parameters of the code large language model are optimized according to the loss to obtain the trained code large language model. The label of the training data is the result of processing the third training data according to the event extraction task, which is marked in advance for the third training data.

[0046] The code language model trained with the third training data is fine-tuned by the Low-Rank Adaptation method to obtain a third code language model, which is a code language model corresponding to the named entity relationship extraction task. The third code language model is saved in the model library.

[0047] Furthermore, the large code language model trained with the training data corresponding to various tasks is fine-tuned by the Low-Rank Adaptation method to obtain the large code language model corresponding to various tasks, including: for various tasks: constructing a lora branch for the target network layer in the large code language model trained with the training data corresponding to the task; inputting the training data corresponding to the task into the large code language model after constructing the lora branch, and inside the large code language model after constructing the lora branch: processing the output of the previous network layer of the target network layer through the lora branch, adding the output of the target network layer to the output of the lora branch as the input of the next network layer of the target network layer, wherein the output of the lora branch is the result of the lora branch processing the output of the previous network layer of the target network layer.

[0048] The target network layer in the code large language model can be the embedding layer, multi-head self-attention network, feedforward network, layer normalization and fully connected layer in the code large language model. The code large language model is also composed of networks such as embedding layer, multi-head self-attention network, feedforward network, layer normalization and fully connected layer.

[0049] To construct a lora branch for the target network layer, the first matrix and the second matrix connected in sequence are used as lora branches. The first matrix obeys the normal distribution after initialization, and the second matrix is ​​initialized as a zero matrix. The output of the previous layer of the target network layer is processed through the lora branch, and the output of the target network layer is added to the output of the lora branch as the input of the next layer of the target network layer.

[0050] Assuming that the parameter matrix of the target network layer is d*d, the first matrix is ​​r*d, and the second matrix is ​​d*r. Then the output of the previous network layer of the target network layer is processed by the first matrix and the second matrix in turn, and the output of the obtained lora branch is also d*d. Therefore, the output of the target network layer and the output of the lora branch can be directly added as the input of the next network layer of the target network layer.

[0051] All the above-mentioned optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0052] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0053] Figure 3 Schematic diagram of a data processing device based on a code language model provided by an embodiment of the present disclosure. Figure 3 As shown, the data processing device based on the code language model includes:

[0054] The first acquisition module 301 is configured to acquire the data to be processed and the target task corresponding to the data to be processed;

[0055] An extraction module 302 is configured to extract prompt guidance information of the data to be processed according to the target task using the large language model;

[0056] The conversion module 303 is configured to convert the data to be processed and the prompt guidance information into a data code and a prompt code in a preset format respectively;

[0057] The second acquisition module 304 is configured to acquire the target code language model corresponding to the target task from the model library;

[0058] The processing module 305 is configured to process the data code according to the prompt code using the target code language model to obtain a processing result.

[0059] According to the technical solution provided by the embodiment of the present application, the data to be processed and the target task corresponding to the data to be processed are obtained; the prompt guidance information of the data to be processed is extracted according to the target task using the large language model; the data to be processed and the prompt guidance information are respectively converted into data code and prompt code in a preset format; the target code large language model corresponding to the target task is obtained from the model library; the data code is processed according to the prompt code using the target code large language model to obtain the processing result. The above technical means can solve the problem of low efficiency and accuracy of information extraction in the prior art, thereby improving the efficiency and accuracy of information extraction.

[0060] In some embodiments, the extraction module 302 is further configured to use the large language model to extract training prompt guidance information of the training data corresponding to the target task according to the target task; calculate the loss between the training prompt guidance information and the label of the training data, optimize the model parameters of the large language model based on the loss, and complete the training of the large language model.

[0061] In some embodiments, the second acquisition module 304 is configured to acquire training data corresponding to various tasks; train the code large language model using the training data corresponding to various tasks; fine-tune the code large language model trained with the training data corresponding to various tasks by the Low-Rank Adaptation method to obtain the code large language model corresponding to various tasks; and save the code large language model corresponding to various tasks into the model library.

[0062] In some embodiments, the second acquisition module 304 is configured to, for various tasks: use the large language model to extract the training prompt guidance information of the training data corresponding to the task according to the task; convert the training data and the training prompt guidance information into training data codes and training prompt codes in preset formats respectively; use the code large language model to process the training data codes according to the training prompt codes to obtain the training processing results; calculate the loss between the training processing results and the labels of the training data, optimize the model parameters of the code large language model according to the loss, and obtain the code large language model corresponding to the task.

[0063] In some embodiments, the second acquisition module 304 is configured to acquire first training data corresponding to the named entity recognition task; train the code large language model using the first training data; fine-tune the code large language model trained with the first training data by the Low-Rank Adaptation method to obtain a first code large language model; and save the first code large language model into the model library.

[0064] In some embodiments, the second acquisition module 304 is configured to extract the training prompt guidance information of the first training data according to the named entity recognition task using the large language model; convert the training data and the training prompt guidance information into a training data code and a training prompt code in a preset format respectively; use the code large language model to process the training data code according to the training prompt code to obtain a training processing result; calculate the loss between the training processing result and the label of the first training data, optimize the model parameters of the code large language model according to the loss, and obtain the trained code large language model. The label of the first training data is the result of processing the first training data according to the named entity recognition task that is marked in advance for the first training data.

[0065] In some embodiments, the second acquisition module 304 is configured to acquire second training data corresponding to the named entity relationship extraction task; train the code large language model using the second training data; fine-tune the code large language model trained with the second training data by the Low-Rank Adaptation method to obtain a second code large language model; and save the second code large language model into the model library.

[0066] In some embodiments, the second acquisition module 304 is configured to use the large language model to extract the training prompt guidance information of the second training data according to the named entity relationship extraction task; convert the training data and the training prompt guidance information into a training data code and a training prompt code in a preset format respectively; use the code large language model to process the training data code according to the training prompt code to obtain a training processing result; calculate the loss between the training processing result and the label of the second training data, optimize the model parameters of the code large language model according to the loss, and obtain a trained code large language model. The label of the second training data is the result of processing the second training data according to the named entity relationship extraction task that is marked in advance for the second training data.

[0067] In some embodiments, the second acquisition module 304 is configured to acquire third training data corresponding to the event extraction task; train the code large language model using the third training data; fine-tune the code large language model trained with the third training data by the Low-Rank Adaptation method to obtain a third code large language model; and save the third code large language model into the model library.

[0068] In some embodiments, the second acquisition module 304 is configured to train the code large language model using the third training data: extract the training prompt guidance information of the third training data using the large language model according to the event extraction task; convert the training data and the training prompt guidance information into a training data code and a training prompt code in a preset format respectively; use the code large language model to process the training data code according to the training prompt code to obtain a training processing result; calculate the loss between the training processing result and the label of the third training data, optimize the model parameters of the code large language model according to the loss, and obtain the trained code large language model. The label of the training data is the result of processing the third training data according to the event extraction task that is marked in advance for the third training data.

[0069] In some embodiments, the second acquisition module 304 is configured to, for various tasks: construct a lora branch for the target network layer in the code large language model trained with the training data corresponding to the task; input the training data corresponding to the task into the code large language model after the lora branch is constructed, and inside the code large language model after the lora branch is constructed: process the output of the previous network layer of the target network layer through the lora branch, and add the output of the target network layer to the output of the lora branch as the input of the next network layer of the target network layer, wherein the output of the lora branch is the result of the lora branch processing the output of the previous network layer of the target network layer.

[0070] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0071] Figure 4 FIG. 4 is a schematic diagram of an electronic device 4 provided in an embodiment of the present disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and capable of running on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0072] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , which may include more or fewer components than those shown in the figure, or different components.

[0073] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0074] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, the electronic device 4 is equipped with a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. The memory 402 may also include both an internal storage unit of the electronic device 4 and an external storage device. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0075] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above-mentioned functional units and module divisions are used as examples for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. In the embodiment, each functional unit and module can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0076] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the above-mentioned various method embodiment steps when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0077] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure.

Claims

1. A data processing method based on a code language model, characterized in that: include: Acquire the data to be processed and the target task corresponding to the data to be processed, wherein the target task includes a named entity recognition task, a named entity relationship extraction task and an event extraction task; Extracting prompt guidance information of the data to be processed according to the target task using the first language model; Converting the data to be processed and the prompt guidance information into a data code and a prompt code in a preset format respectively; Acquire a target code large language model corresponding to the target task from a model library, wherein the target code large language model and the first large language model are the same model, the first large language model is trained to understand and generate natural language text, and the target code large language model is trained to understand and generate a code in a preset format corresponding to the natural language text; Processing the data code according to the prompt code using the target code large language model to obtain a processing result; Before extracting the prompt guidance information of the data to be processed according to the target task using the first large language model, the method further includes: extracting the training prompt guidance information of the training data corresponding to the target task according to the target task using the first large language model; calculating the loss between the training prompt guidance information and the label of the training data, optimizing the model parameters of the first large language model according to the loss, and completing the training of the first large language model; Before obtaining the target code language model corresponding to the target task from the model library, the method further includes: obtaining training data corresponding to various tasks; training the code language model using the training data corresponding to various tasks; fine-tuning the code language model trained with the training data corresponding to various tasks by the Low-Rank Adaptation method to obtain the code language model corresponding to various tasks; and saving the code language model corresponding to various tasks to the model library; Among them, the large code language model trained with the training data corresponding to various tasks is fine-tuned by the Low-Rank Adaptation method to obtain the large code language model corresponding to various tasks, including: for various tasks: constructing a lora branch for the target network layer in the large code language model trained with the training data corresponding to the task; inputting the training data corresponding to the task into the large code language model after constructing the lora branch, and inside the large code language model after constructing the lora branch: processing the output of the upper network layer of the target network layer through the lora branch, adding the output of the target network layer to the output of the lora branch as the input of the next network layer of the target network layer, wherein the output of the lora branch is the result of the lora branch processing the output of the upper network layer of the target network layer.

2. The method according to claim 1, characterized in that: The method further comprises: Obtaining first training data corresponding to a named entity recognition task; Using the first training data to train a code language model; Fine-tune the code language model trained with the first training data by a Low-Rank Adaptation method to obtain a first code language model; The first code language model is saved in the model library.

3. The method according to claim 1, characterized in that: The method further comprises: Obtaining second training data corresponding to the named entity relationship extraction task; Using the second training data to train a code language model; Fine-tune the code language model trained with the second training data by a Low-Rank Adaptation method to obtain a second code language model; The second code language model is saved in the model library.

4. The method according to claim 1, characterized in that: The method further comprises: Obtain third training data corresponding to the event extraction task; Using the third training data to train a code language model; Fine-tune the code language model trained with the third training data by a Low-Rank Adaptation method to obtain a third code language model; The third code language model is saved in the model library.

5. The method according to claim 1, characterized in that: The code language model is trained using training data corresponding to various tasks, including: For various tasks: Extracting training prompt guidance information of training data corresponding to the task according to the task using the first language model; Converting the training data and the training prompt guidance information into the training data code and the training prompt code in the preset format respectively; Processing the training data code according to the training prompt code using a code language model to obtain a training processing result; The loss between the training processing result and the label of the training data is calculated, and the model parameters of the code large language model are optimized according to the loss to obtain the code large language model corresponding to the task.

6. A data processing device based on a code language model, characterized in that: include: A first acquisition module is configured to acquire data to be processed and target tasks corresponding to the data to be processed, wherein the target tasks include named entity recognition tasks, named entity relationship extraction tasks, and event extraction tasks; An extraction module is configured to extract prompt guidance information of the data to be processed according to the target task using the first language model; A conversion module, configured to convert the data to be processed and the prompt guidance information into a data code and a prompt code in a preset format respectively; A second acquisition module is configured to acquire a target code large language model corresponding to the target task from a model library, wherein the target code large language model and the first large language model are the same model, the first large language model is trained to understand and generate natural language text, and the target code large language model is trained to understand and generate a code in a preset format corresponding to the natural language text; A processing module is configured to process the data code according to the prompt code using the target code language model to obtain a processing result; The extraction module is further configured to extract training prompt guidance information of the training data corresponding to the target task using the first large language model according to the target task; calculate the loss between the training prompt guidance information and the label of the training data, optimize the model parameters of the first large language model according to the loss, and complete the training of the first large language model; The second acquisition module is further configured to acquire training data corresponding to various tasks; train the code language model using the training data corresponding to various tasks; fine-tune the code language model trained with the training data corresponding to various tasks by the Low-Rank Adaptation method to obtain the code language model corresponding to various tasks; and save the code language model corresponding to various tasks into the model library; The second acquisition module is also configured to, for various tasks: construct a lora branch for the target network layer in the code large language model trained with the training data corresponding to the task; input the training data corresponding to the task into the code large language model after constructing the lora branch, and inside the code large language model after constructing the lora branch: process the output of the previous network layer of the target network layer through the lora branch, and add the output of the target network layer to the output of the lora branch as the input of the next network layer of the target network layer, wherein the output of the lora branch is the result of the lora branch processing the output of the previous network layer of the target network layer.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images