Evaluation model training method, data processing method and device

Abstract

The embodiment of the invention discloses an evaluation model training method. According to the method, a standard operation program text is disassembled into function points, then each function point is realized in a code mode and in a mode of calling a large model, the two realization modes are tested by utilizing test data, and applicability indexes of the two realization modes to the function points are determined based on test results. And utilizing the applicability index data to train an evaluation model which can quantitatively judge whether the function point is suitable for being realized by using a code or a large model according to the description of the function point. The evaluation model training device and the data processing method and device provided by the embodiment of the specification also have the above beneficial effects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular, to an evaluation model training method and a data processing method and device. BACKGROUND

[0002] A standard operating procedure (SOP) is a document that describes a job and its workflow and steps in detail. When implementing a standard operating procedure (SOP) through a system, the standard operating procedure (SOP) needs to be converted into programmatic business logic (PBL) first. There are mainly two ways to convert the standard operating procedure into programmatic business logic, one is to implement it through python, java and other code methods, and the other is to implement it through a large language model, but both methods have their limitations.

[0003] Therefore, in the process of converting the standard operating procedure into programmatic business logic, the main difficulty lies in subtask splitting, that is, which function points / subtasks are suitable for implementation using python, java and other code methods, and which function points are suitable for directly using natural language as a prompt to call a large model. The traditional method completely relies on human experience and cannot achieve automatic splitting, and the effect of non-splitting cannot be effectively guaranteed. SUMMARY

[0004] One or more embodiments of the present specification provide an evaluation model training method and a data processing method and device to at least partially solve the above technical problems.

[0005] In a first aspect, an evaluation model training method is provided, comprising: constructing a function point set based on a pre-collected standard operating procedure text, collecting test data for each function point in the function point set; for each function point in the function point set, respectively implementing the function of the function point through a code method and a method of calling a large model, and testing the functions implemented by the two implementation methods respectively using the test data of the function point, to obtain test results; based on the test results, determining the applicability indicators of the two implementation methods for the function point; generating a function description text for each function point, taking the function description text of the function point and the applicability indicators of the two implementation methods for the function point as training samples, fine-tuning a target large model, and obtaining the evaluation model.

[0006] As an optional implementation of the method of the first aspect, constructing a function point set based on a pre-collected standard operating procedure text specifically comprises: Splitting the standard operating procedure text sentence by sentence to obtain a candidate function point set; De-duplicating the candidate function point set to obtain the function point set.

[0007] As an optional implementation of the method of the first aspect, for each function point in the function point set, the function of the function point is implemented by code and by calling a large model respectively, and the test data of the function point is used to test the functions implemented by the two implementation methods respectively to obtain test results, specifically including: For each function point in the function point set, generate code for implementing the function of the function point, and run the code to process the test data of the function point to obtain a first test result; For each function point in the function point set, use a test large model to process the test data of the function point based on the function of the function point to obtain a second test result.

[0008] Specifically, for each function point in the function point set, code for implementing the function of the function point is generated, specifically including: Construct a prompt word according to the function of the function point; Input the prompt word into a generation large model to obtain code for implementing the function of the function point.

[0009] Specifically, based on the test results, determine the applicability indicators of the two implementation methods for the function point, specifically including: Based on the first test result of the function point and the standard reference result of the test data of the function point, determine the applicability indicator of the function point implemented by code; Based on the second test result of the function point and the standard reference result of the test data of the function point, determine the applicability indicator of the function point implemented by calling a large model.

[0010] As an optional implementation of the method of the first aspect, generate a function description text of each function point, use the function description text of the function point and the applicability indicators of the two implementation methods for the function point as training samples, and fine-tune a target large model to obtain the evaluation model, specifically including: Input the function description text of the function point, the two implementation methods and the applicability indicators of the function point on the two implementation methods into the target large model to fine-tune the target large model.

[0011] As an optional implementation of the method of the first aspect, the function description text of each function point is generated, the function description text of the function point and the applicability indicators of the two implementations to the function point are taken as training samples, the target large model is fine-tuned, and the evaluation model is obtained. Specifically, the evaluation model is obtained by: The function description text of the function point and the two implementations are input into the target large model, and the target large model is driven to evaluate the applicability of the two implementations to the function point respectively. According to the evaluation results of the target large model and the applicability indicators of the function point in the two implementations, an incentive signal is constructed. The incentive signal is fed back to the target large model to fine-tune the target large model.

[0012] In a second aspect, a data processing method is provided, comprising: Obtaining a standard operating procedure text to be processed, and splitting the standard operating procedure text to be processed into function points; Constructing a function description text of the function point; The function description text of the function point is input into an evaluation model, and the evaluation model is driven to evaluate the applicability of the function point implemented by code and the applicability of the function point implemented by calling a large model respectively; the evaluation model is pre-trained by using the evaluation model training method described above; According to the evaluation results output by the evaluation model, the implementation mode of the function point is determined, and the function point is converted into programmatic business logic based on the implementation mode.

[0013] In a third aspect, an evaluation model training device is provided, comprising: A first data acquisition module is configured to acquire a standard operating procedure text. A first function point disassembly module is configured to construct a function point set based on the standard operating procedure text. A second data acquisition module is configured to collect test data for each function point in the function point set. A test module is configured to implement the function of each function point in the function point set by code and by calling a large model respectively, and test the functions implemented by the two implementations respectively using the test data of the function point, to obtain test results. An indicator generation module is configured to determine the applicability indicators of the two implementations to the function point based on the test results. The training module is configured to generate a function description text of each function point, take the function description text of the function point and an applicability index of the function point on the two implementation manners as training samples, fine-tune a target large model, and obtain the evaluation model.

[0014] As an optional implementation of the device of the third aspect, the first function point disassembling module is specifically configured to: disassemble the standard operating procedure text sentence by sentence to obtain a candidate function point set; perform deduplication processing on the candidate function point set to obtain the function point set.

[0015] As an optional implementation of the device of the third aspect, the testing module is specifically configured to: for each function point in the function point set, generate a code for implementing a function of the function point, run the code to process test data of the function point, and obtain a first test result; for each function point in the function point set, use a test large model to process test data of the function point based on a function of the function point, and obtain a second test result.

[0016] Specifically, the testing module is specifically configured to: construct a prompt word according to the function of the function point; input the prompt word into a generation large model to obtain the code for implementing the function of the function point.

[0017] Specifically, the index generation module is specifically configured to: determine an applicability index of the function point implemented by a code-based manner based on the first test result of the function point and a standard reference result of the test data of the function point; determine an applicability index of the function point implemented by a manner of calling a large model based on the second test result of the function point and the standard reference result of the test data of the function point.

[0018] As an optional implementation of the device of the third aspect, the training module is specifically configured to: input the function description text of the function point, the two implementation manners, and the applicability index of the function point on the two implementation manners into the target large model to fine-tune the target large model.

[0019] As an optional implementation of the device of the third aspect, the training module is specifically configured to: The function description text of the function point and the two implementation manners are input into the target large model, and the target large model is driven to respectively evaluate the applicability of the two implementation manners to the function point. According to the evaluation result of the target large model and the applicability indicators of the function point in the two implementation manners, an excitation signal is constructed. The excitation signal is fed back to the target large model to fine-tune the target large model.

[0020] In a fourth aspect, a data processing apparatus is provided, comprising: A third data acquisition module is configured to acquire a standard operation procedure text to be processed. A second function point decomposition module is configured to split the standard operation procedure text to be processed into function points. An evaluation module is configured to construct a function description text of the function point, input the function description text of the function point into an evaluation model, and drive the evaluation model to respectively evaluate the applicability of the function point implemented through a code manner and the applicability of the function point implemented through calling a large model. The evaluation model is pre-trained by using the evaluation model training method described above. A processing module is configured to determine an implementation manner of the function point according to an evaluation result output by the evaluation model, and convert the function point into a programmatic business logic based on the implementation manner.

[0021] In a fifth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed on an electronic device, the electronic device is caused to execute the evaluation model training method described above, or execute the data processing method described above.

[0022] In a sixth aspect, an electronic device is provided, comprising: At least one memory is configured to store a program. At least one processor is configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the evaluation model training method described above, or execute the data processing method described above.

[0023] The evaluation model training method described in the embodiments of the present specification has the following beneficial effects: The method disassembles the standard operating procedure text into function points, then implements each function point through code and through calling a large model respectively, tests the two implementation manners by using test data, and determines the applicability indexes of the two implementation manners for the function point based on the test results. By using the applicability index data, an evaluation model capable of quantitatively judging whether the function point is suitable for code implementation or large model implementation according to the function point description is trained.

[0024] When task splitting is performed on a new standard operating procedure text, the evaluation model can be used to automatically evaluate the implementation manner of the function point, so that the new standard operating procedure text can be automatically split according to the evaluation result of the evaluation model, and the splitting effect is more guaranteed.

[0025] The evaluation model training apparatus, data processing method and apparatus described in the embodiments of the present specification also have the above beneficial effects. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0027] Figure 1 An exemplary flowchart of an evaluation model training method is shown.

[0028] Figure 2 An exemplary flowchart of a data processing method is shown.

[0029] Figure 3 An exemplary structure diagram of an evaluation model training apparatus is shown.

[0030] Figure 4 An exemplary structure diagram of a data processing apparatus is shown.

[0031] Figure 5 An exemplary structure diagram of an electronic device is shown. DETAILED DESCRIPTION

[0032] It should be noted that the terms used in the embodiments of the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a," "said," and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0033] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the embodiments of the specification will be described clearly and completely in the following with reference to the drawings in the embodiments of the specification. Obviously, the described embodiments are only a part of the embodiments of the specification, not all. Therefore, those skilled in the art should realize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, in order to be clear and concise, the description in the following description omits the description of well-known functions and structures.

[0034] It should be noted that the steps of the corresponding method in other embodiments are not necessarily performed in the order shown and described in the specification. In some other embodiments, the steps included in the method can be more or less than described in the specification. In addition, a single step described in the specification can be divided into multiple steps for description in other embodiments; and multiple steps described in the specification can be combined into a single step for description in other embodiments.

[0035] One or more embodiments of the present specification propose an evaluation model training method, a data processing method and device.

[0036] The evaluation model training method, data processing method and device described in one or more embodiments of the present specification will be further described in detail below with reference to the drawings and specific embodiments of the specification, but this detailed description does not constitute a limitation on the embodiments of the present specification.

[0037] Please refer to Figure 1 , Figure 1 A flowchart of an evaluation model training method is proposed in one or more embodiments of the present specification. The method aims to train an evaluation model that can quantitatively judge whether a function point is suitable for code implementation or suitable for large model implementation according to the function point description.

[0038] It should be noted that the evaluation model training method can be implemented by relying on the evaluation model training device described in one or more embodiments of the present specification, but is not limited to the device. As Figure 1 shown, the method includes steps S100 to S106.

[0039] S100: Construct a function point set based on a pre-collected standard operating procedure text, and collect test data for each function point in the function point set.

[0040] A standard operating procedure (SOP) refers to a standard operating procedure, which is a file that describes work and its workflow and steps in detail. It is a standard process specification for internal transactions of an organization, usually containing detailed processes of a specific business, product or service, for internal personnel to refer to and follow when operating. The content of SOP mainly includes standardized processes, clear responsibilities and authorities, emergency measures, quality standards and evaluation, document management, etc.

[0041] The function point in this step refers to the result of granularizing and disassembling the standard operating procedure text. It should be noted that the degree of granulation can be set according to the needs during the granularization and disassembly of the standard operating procedure text, which is not limited in this embodiment. For example, the above-mentioned function point can be a process node, an input / output node, a decision point, etc.

[0042] In some embodiments, the above-mentioned standard operating procedure text can be split sentence by sentence to obtain a candidate function point set, i.e. each sentence in the standard operating procedure text is taken as a function point.

[0043] Considering that the candidate function points obtained by splitting may overlap, the candidate function point set obtained by splitting can be de-duplicated to obtain a function point set.

[0044] For each function point in the function point set, test data is collected. The test data is a set of data used to verify whether the logical function of the function point is correct, whether the performance is stable, etc. For example, the function of a certain function point is to filter out emails containing fraud information from emails, and the test data of the function point is a plurality of email content data with artificial annotation (annotating whether the email is a fraud email).

[0045] S102: For each function point in the function point set, the function of the function point is implemented by code and by calling a large model respectively, and the test data of the function point is used to test the functions implemented by the two implementation methods respectively to obtain test results.

[0046] Formal programming languages (such as Python, java, etc.) define deterministic task decomposition, steps, control flow (sequential, parallel, conditional), data flow, and error handling logic, which are suitable for describing the skeleton of business logic that can be accurately executed. Natural language instruction expression is more flexible and has more rich semantic connotations, which is suitable for describing parts of business logic with uncertainty or difficult to accurately define, mainly for large models to process.

[0047] To test whether each function point is suitable for the code mode or the large model calling mode, the same test data can be used to test the two implementation modes respectively: For each function point in the function point set, code for implementing the function of the function point is generated, and the code is run to process the test data of the function point to obtain a first test result. Specifically, some large models can be randomly selected, prompt words are constructed according to the function of the function point, and the prompt words are input into the large model to obtain code for implementing the function of the function point.

[0048] For each function point in the function point set, the test large model processes the test data of the function point based on the function of the function point to obtain a second test result.

[0049] S104: Based on the test results, determine the applicability indicators of the two implementation modes for the function points.

[0050] The above-mentioned applicability indicators can be set according to requirements, such as selecting recall rate, accuracy rate, etc. as the applicability indicators. The selection and number of applicability indicators can be set according to requirements, and the present embodiment does not limit this.

[0051] Specifically, based on the first test result of the function point and the standard reference result of the test data of the function point, the applicability indicator of the function point implemented by the code mode can be determined.

[0052] Based on the second test result of the function point and the standard reference result of the test data of the function point, the applicability indicator of the function point implemented by calling the large model can be determined.

[0053] The above-mentioned standard reference result can be specific information artificially annotated to evaluate whether the first test result and the second test result are correct.

[0054] S106: Generate a function description text of each function point, use the function description text of the function point and the applicability indicators of the function point for the two implementation modes as training samples, fine-tune the target large model to obtain an evaluation model.

[0055] The target large model here can be a large language model randomly selected.

[0056] When fine-tuning the target large model, the function description text of the function point, the two implementation modes (implemented by code and implemented by large model) and the applicability indicators of the function point in the two implementation modes can be input into the target large model to fine-tune the target large model.

[0057] Alternatively, the target large model can also be fine-tuned in a reinforcement learning manner. Specifically, the functional description text of the function point and the two implementation manners (through code implementation and through large model implementation) can be input into the target large model, and the target large model can be driven by the prompt word to respectively evaluate the applicability of the two implementation manners to the function point. The evaluation index needs to be consistent with the above-mentioned applicability index, that is, if the correctness rate is used as the applicability index, the correctness rate is also used as the evaluation index. According to the evaluation result of the target large model and the applicability index of the function point in the two implementation manners, a stimulus signal is constructed. The stimulus signal is fed back to the target large model to fine-tune the target large model.

[0058] The above is the evaluation model training method described in one or more embodiments of the present specification. The method disassembles the standard operating procedure text into function points, then respectively implements each function point through code and through calling a large model, tests the two implementation manners using test data, and determines the applicability index of the two implementation manners to the function point based on the test result. Using these applicability index data, an evaluation model is trained, which can quantitatively judge whether the function point is suitable for code implementation or large model implementation based on the function point description.

[0059] Corresponding to the above-mentioned evaluation model training method, one or more embodiments of the present specification propose a data processing method. As shown in Figure 2 The method includes steps S200 to S206.

[0060] S200: Acquire the standard operating procedure text to be processed, and split the standard operating procedure text to be processed into function points.

[0061] S202: Construct a functional description text of the function point.

[0062] S204: Input the functional description text of the function point into the evaluation model, and drive the evaluation model to respectively evaluate the applicability of the function point through code implementation and the applicability of the function point through calling a large model.

[0063] S206: Determine the implementation manner of the function point according to the evaluation result output by the evaluation model, and convert the function point into programmatic business logic based on the implementation manner.

[0064] In the above-mentioned data processing method, the evaluation model used is pre-trained by the above-mentioned evaluation model training method. The training process and working principle of the evaluation model have been described in detail in the above-mentioned evaluation model training method, and will not be repeated here.

[0065] The data processing method can realize automatic evaluation of the function point implementation manner by using the evaluation model when task splitting is performed on the new standard operating procedure text, so that the new standard operating procedure text can be automatically split according to the evaluation result of the evaluation model, and the splitting effect is more guaranteed.

[0066] Corresponding to the above-mentioned evaluation model training method, an evaluation model training device is provided in one or more embodiments of the present specification. As shown in Figure 3 the evaluation model training device comprises: The first data acquisition module 301 is configured to acquire a standard operating procedure text.

[0067] The first function point disassembly module 302 is configured to construct a function point set based on the standard operating procedure text.

[0068] The second data acquisition module 303 is configured to collect test data for each function point in the function point set.

[0069] The test module 304 is configured to implement the function of the function point by code and by calling a large model for each function point in the function point set, respectively, and test the functions implemented by the two implementation manners by using the test data of the function point, to obtain test results.

[0070] The index generation module 305 is configured to determine the applicability index of the two implementation manners for the function point based on the test results.

[0071] The training module 306 is configured to generate a function description text of each function point, use the function description text of the function point and the applicability index of the function point in the two implementation manners as a training sample, fine-tune the target large model, and obtain an evaluation model.

[0072] Optionally, the first function point disassembly module 302 is specifically configured to: split the standard operating procedure text sentence by sentence to obtain a candidate function point set; and perform deduplication processing on the candidate function point set to obtain the function point set.

[0073] Optionally, the test module 304 is specifically configured to: For each function point in the function point set, generate code for implementing the function of the function point, run the code to process the test data of the function point, and obtain a first test result; For each function point in the function point set, use the test large model to process the test data of the function point based on the function of the function point, and obtain a second test result.

[0074] Specifically, the aforementioned test module 304 is also used to construct prompt words based on the function of the function point; input the prompt words to generate a large model, and obtain the code used to implement the function of the function point.

[0075] Specifically, the aforementioned indicator generation module 305 is used for: Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by code is determined; Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

[0076] Optionally, the training module 306 described above can be used to fine-tune the target large model in the following ways: Input the functional description text of the function point, two implementation methods, and the applicability index of the function point in the two implementation methods into the target large model to fine-tune the target large model.

[0077] Alternatively, the training module 306 described above can also be used to fine-tune the target large model in the following way: Input the functional description text of the function point and the two implementation methods into the target large model, and drive the target large model to evaluate the applicability of the two implementation methods to the function point respectively; Based on the evaluation results of the target model and the applicability indicators of this function in the two implementation methods, an incentive signal is constructed. The excitation signal is fed back to the target large model to fine-tune the target large model.

[0078] Below, with Figure 3 The implementation principle of this device is illustrated using the evaluation model training device shown as an example. Taking a module as a software functional unit as an example, the first data acquisition module 301 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, the first data acquisition module 301 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically proximate data centers. Typically, a region may include multiple AZs.

[0079] Likewise, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, usually one VPC is set in one region, and communication between two VPCs in the same region or between VPCs in different regions needs to set a communication gateway in each VPC to realize the interconnection between VPCs through the communication gateway.

[0080] As an example of a hardware functional unit, the first data acquisition module 301 can include at least one computing device, such as a server, etc. Alternatively, the first data acquisition module 301 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. Among them, the above-mentioned PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0081] The multiple computing devices included in the first data acquisition module 301 can be distributed in the same region or in different regions. The multiple computing devices included in the first data acquisition module 301 can be distributed in the same AZ or in different AZs. Likewise, the multiple computing devices included in the first data acquisition module 301 can be distributed in the same VPC or in multiple VPCs. Among them, the multiple computing devices can be any combination of server, ASIC, PLD, CPLD, FPGA, and GAL computing devices.

[0082] In other embodiments, the first data acquisition module 301 can be configured to perform any of the steps of the above-described evaluation model training method, the first function point decomposition module 302 can be configured to perform any of the steps of the above-described evaluation model training method, the second data acquisition module 303 can be configured to perform any of the steps of the above-described evaluation model training method, the test module 304 can be configured to perform any of the steps of the above-described evaluation model training method, the index generation module 305 can be configured to perform any of the steps of the above-described evaluation model training method, and the training module 306 can be configured to perform any of the steps of the above-described evaluation model training method. The steps to be implemented by the first data acquisition module 301, the first function point decomposition module 302, the second data acquisition module 303, the test module 304, the index generation module 305, and the training module 306 can be specified as needed, and the entire function of the above-described evaluation model training device can be implemented by the first data acquisition module 301, the first function point decomposition module 302, the second data acquisition module 303, the test module 304, the index generation module 305, and the training module 306 respectively implementing different steps of the above-described evaluation model training method.

[0083] In the present implementation, the device can also be applied to a computer, a server, or the like computing device, or to a computing device cluster comprising at least one computing device, to implement a specific evaluation model training.

[0084] Corresponding to the above-described evaluation model training method, one or more embodiments of the present specification propose a data processing device. As shown in Figure 4 The data processing device comprises: A third data acquisition module 401 is configured to acquire a standard operating procedure text to be processed.

[0085] A second function point decomposition module 402 is configured to split the standard operating procedure text to be processed into function points.

[0086] An evaluation module 403 is configured to construct a function description text of the function points, input the function description text of the function points into an evaluation model, and drive the evaluation model to evaluate the applicability of the function points implemented through code and the applicability of the function points implemented through calling a large model, respectively. The evaluation model is pre-trained using the above-described evaluation model training method.

[0087] A processing module 404 is configured to determine an implementation manner of the function points according to an evaluation result output by the evaluation model, and convert the function points into programmatic business logic based on the implementation manner.

[0088] In the following, Figure 4The data processing apparatus shown is used to illustrate the implementation principle of the apparatus. Taking a module as an example of a software functional unit, the third data acquisition module 401 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the third data acquisition module 401 can include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code can be distributed in the same region, or distributed in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ), or distributed in different AZs, each AZ including a data center or multiple data centers in close geographical proximity. Generally, one region can include multiple AZs.

[0089] Similarly, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC), or distributed in multiple VPCs. Generally, one VPC is set up in one region, and communication between two VPCs in the same region and between VPCs in different regions requires setting up a communication gateway in each VPC to achieve interconnection between VPCs through the communication gateway.

[0090] Taking a module as an example of a hardware functional unit, the third data acquisition module 401 can include at least one computing device, such as a server, etc. Alternatively, the third data acquisition module 401 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. The PLD can be implemented by a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0091] The plurality of computing devices included in the third data acquisition module 401 can be distributed in the same region or in different regions. The plurality of computing devices included in the third data acquisition module 401 can be distributed in the same AZ or in different AZs. Similarly, the plurality of computing devices included in the third data acquisition module 401 can be distributed in the same VPC or in multiple VPCs. The plurality of computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0092] In other embodiments, the third data acquisition module 401 can be configured to perform any of the steps of the data processing method described above, the second function point decomposition module 402 can be configured to perform any of the steps of the data processing method described above, the evaluation module 403 can be configured to perform any of the steps of the data processing method described above, and the processing module 404 can be configured to perform any of the steps of the data processing method described above. The steps implemented by the third data acquisition module 401, the second function point decomposition module 402, the evaluation module 403, and the processing module 404 can be specified as needed, and the entire function of the data processing device described above can be implemented by the third data acquisition module 401, the second function point decomposition module 402, the evaluation module 403, and the processing module 404 implementing different steps of the data processing method described above.

[0093] In this implementation, the device can also be applied to a computer, a server, or other computing device, or to a computing device cluster including at least one computing device, to implement a specific data processing method.

[0094] One or more embodiments in the specification also provide an electronic device. Please refer to Figure 5 The electronic device includes a bus 501, a processor 502, a memory 503, and a communication interface 504. The processor 502, the memory 503, and the communication interface 504 communicate through the bus 501. The electronic device can be a server or a terminal device. It should be understood that the number of processors and memories in the electronic device is not limited.

[0095] The bus 501 can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 5Only one bus is shown for simplicity, but there can be more than one bus, and the bus can be of various types, including an address bus, a data bus, a control bus, etc. The bus 501 can include a path for communicating information among the various components of the electronic device, such as the processor 502, the memory 503, and the communication interface 504.

[0096] The processor 502 can include any one or more of a processor CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0097] The memory 503 can include a volatile memory, such as a random access memory (RAM), and a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0098] The memory 503 stores executable program codes, and the processor 502 executes the executable program codes to implement the aforementioned evaluation model training method, or to implement the aforementioned data processing method.

[0099] The communication interface 504 uses a transceiver module, such as but not limited to a network interface card or a transceiver, to enable communication between the electronic device and other devices or communication networks.

[0100] One or more embodiments in the specification provide a computer-readable storage medium storing a computer program, which, when executed on an electronic device, causes the electronic device to perform the aforementioned evaluation model training method, or to perform the aforementioned data processing method.

[0101] The computer-readable storage medium can be any available medium or a data center including one or more available media that the electronic device can store. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk), etc. The computer-readable storage medium includes instructions that instruct the electronic device to perform the aforementioned evaluation model training method, or to perform the aforementioned data processing method.

[0102] It can be understood that the structural examples in the specification do not constitute a specific limitation on the system of the embodiments of the specification. In other embodiments of the specification, the above-mentioned system can include more or less components than the examples, or combine certain components, or split certain components, or different component arrangement. The components illustrated can be implemented in hardware, software or a combination of software and hardware.

[0103] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0104] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing are also possible or advantageous.

[0105] It should be noted that the above only lists specific embodiments of the present application, and obviously the present application is not limited to the above embodiments, and there are many similar changes. All variations directly derived or thought of by those skilled in the art from the disclosure of the present application shall fall within the protection scope of the present application.

Claims

1. A method for evaluating model training, comprising: A set of function points is constructed based on pre-collected standard operating procedure texts, and test data is collected for each function point in the set of function points; For each function point in the set of function points, the function point is implemented by means of code and by calling the large model, respectively. The function implemented by the two methods is tested using the test data of the function points, and the test results are obtained. Based on the test results, the applicability index of the two implementation methods for the functional points is determined; Generate a functional description text for each functional point, and use the functional description text and the applicability index of the two implementation methods for the functional point as training samples to fine-tune the target large model and obtain the evaluation model.

2. The method according to claim 1, wherein constructing a set of function points based on pre-collected standard operating procedure text specifically includes: The standard operating procedure text is split sentence by sentence to obtain a set of candidate function points; The candidate function point set is deduplicated to obtain the function point set.

3. The method according to claim 1, for each function point in the set of function points, the function of the function point is implemented by means of code and by calling a large model, respectively, and the function implemented by the two implementation methods is tested using the test data of the function points to obtain test results, specifically including: For each function point in the set of function points, generate code to implement the function of the function point, run the code to process the test data of the function point, and obtain the first test result; For each function point in the set of function points, the test data of the function point is processed using a large test model based on the function of the function point to obtain a second test result.

4. The method according to claim 3, for each function point in the set of function points, generating code to implement the function of the function point, specifically includes: Based on the functions of the aforementioned functional points, prompt words are constructed; The prompt words are input to generate a large model, which yields the code used to implement the function points.

5. The method according to claim 3, based on the test results, determining the applicability index of the two implementation methods for the functional point, specifically includes: Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by code is determined; Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

6. According to claim 1, the method involves generating a functional description text for each functional point, using the functional description text and the applicability index of the two implementation methods for the functional point as training samples, fine-tuning the target large model to obtain the evaluation model, specifically including: The functional description text of the functional point, the two implementation methods, and the applicability index of the functional point in the two implementation methods are input into the target large model to fine-tune the target large model.

7. The method according to claim 1, generating functional description text for each functional point, using the functional description text of the functional point and the applicability index of the two implementation methods for the functional point as training samples, fine-tuning the target large model to obtain the evaluation model, specifically including: The functional description text of the functional point and the two implementation methods are input into the target large model, which drives the target large model to evaluate the applicability of the two implementation methods to the functional point. Based on the evaluation results of the target large model and the applicability indicators of the functional points in the two implementation methods, an incentive signal is constructed; The excitation signal is fed back to the target large model to fine-tune the target large model.

8. A data processing method, comprising: Obtain the standard operating procedure text to be processed, and break the standard operating procedure text to be processed into function points; Construct the functional description text for the aforementioned functional points; Input the functional description text of the functional point into the evaluation model, and drive the evaluation model to evaluate the applicability of the functional point implemented by code and the applicability implemented by calling a large model respectively; The evaluation model is pre-trained using the method described in any one of claims 1 to 7; Based on the evaluation results output by the evaluation model, the implementation method of the function point is determined, and the function point is converted into programmatic business logic based on the implementation method.

9. An evaluation model training device, comprising: The first data acquisition module is used to acquire the standard operating procedure text; The first function point decomposition module is used to construct a set of function points based on the standard operating procedure text; The second data acquisition module is used to collect test data for each function point in the set of function points; The testing module is used to implement the function of each function in the set of function points by means of code and by calling the large model, and to test the function implemented by the two methods using the test data of the function points to obtain test results. The indicator generation module is used to determine the applicability indicators of the two implementation methods for the functional point based on the test results. The training module is used to generate functional description text for each functional point, and uses the functional description text and the applicability index of the functional point in the two implementation methods as training samples to fine-tune the target large model and obtain the evaluation model.

10. The apparatus according to claim 9, wherein the first function point decomposition module is specifically used for: The standard operating procedure text is split sentence by sentence to obtain a set of candidate function points; The candidate function point set is deduplicated to obtain the function point set.

11. The apparatus according to claim 9, wherein the testing module is specifically used for: For each function point in the set of function points, generate code to implement the function of the function point, run the code to process the test data of the function point, and obtain the first test result; For each function point in the set of function points, the test data of the function point is processed using a large test model based on the function of the function point to obtain a second test result.

12. The apparatus according to claim 11, wherein the testing module is specifically used for: Based on the functions of the aforementioned functional points, prompt words are constructed; The prompt words are input to generate a large model, which yields the code used to implement the function points.

13. The apparatus according to claim 11, wherein the index generation module is specifically used for: Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by code is determined; Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

14. The apparatus according to claim 9, wherein the training module is specifically used for: The functional description text of the functional point, the two implementation methods, and the applicability index of the functional point in the two implementation methods are input into the target large model to fine-tune the target large model.

15. The apparatus according to claim 9, wherein the training module is specifically used for: The functional description text of the functional point and the two implementation methods are input into the target large model, which drives the target large model to evaluate the applicability of the two implementation methods to the functional point. Based on the evaluation results of the target large model and the applicability indicators of the functional points in the two implementation methods, an incentive signal is constructed; The excitation signal is fed back to the target large model to fine-tune the target large model.

16. A data processing apparatus, comprising: The third data acquisition module is used to acquire the text of the standard operating procedure to be processed. The second function point decomposition module is used to decompose the standard operating procedure text to be processed into function points; The evaluation module is used to construct the functional description text of the functional point, input the functional description text of the functional point into the evaluation model, and drive the evaluation model to evaluate the applicability of the functional point implemented by code and the applicability implemented by calling a large model respectively. The evaluation model is pre-trained using the method described in any one of claims 1 to 7; The processing module determines the implementation method of the function point based on the evaluation results output by the evaluation model, and converts the function point into programmatic business logic based on the implementation method.

17. A computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the method as claimed in any one of claims 1 to 7, or to perform the method as claimed in claim 8.

18. An electronic device comprising: At least one memory for storing programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as claimed in any one of claims 1 to 7, or to perform the method as claimed in claim 8.

Images