Evaluation model training method, data processing method and device

Abstract

This specification discloses an evaluation model training method. The method decomposes standard operating procedure text into functional points, then implements each functional point both through code and by calling a large model. Test data is then used to test these two implementation methods, and the applicability index of each method for the functional point is determined based on the test results. Using these applicability index data, an evaluation model is trained that can quantitatively determine whether a functional point is more suitable for code implementation or implementation using a large model, based on the functional point description. The evaluation model training device, data processing method, and apparatus described in this specification also have the aforementioned beneficial effects.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an evaluation model training method, data processing method, and apparatus. Background Technology

[0002] A Standard Operating Procedure (SOP) is a document that details a task, its workflow, and steps. When implementing an SOP through a system, it must first be converted into Programmatic Business Logic (PBL). There are two main ways to convert a SOP into PBL: one is through code such as Python or Java, and the other is through a large language model. However, both methods have their limitations.

[0003] Therefore, the main challenge in transforming standard operating procedures into programmatic business logic lies in subtask decomposition, specifically determining which functionalities / subtasks are suitable for implementation using Python, Java, or other code-based methods, and which functionalities are best suited for directly using natural language as prompts to call the larger model. Traditional methods rely entirely on human experience, failing to achieve automated decomposition or guaranteeing effective decomposition. Summary of the Invention

[0004] One or more embodiments of this specification provide an evaluation model training method, a data processing method, and an apparatus to at least partially solve the above-mentioned technical problems.

[0005] Firstly, a method for evaluating model training is provided, including:

[0006] A set of function points is constructed based on pre-collected standard operating procedure texts, and test data is collected for each function point in the set of function points;

[0007] For each function point in the set of function points, the function point is implemented by means of code and by calling the large model, respectively. The function implemented by the two methods is tested using the test data of the function points, and the test results are obtained.

[0008] Based on the test results, the applicability index of the two implementation methods for the functional points is determined;

[0009] Generate a functional description text for each functional point, and use the functional description text and the applicability index of the two implementation methods for the functional point as training samples to fine-tune the target large model and obtain the evaluation model.

[0010] As an optional implementation of the method described in the first aspect, a set of function points is constructed based on pre-collected standard operating procedure text, specifically including:

[0011] The standard operating procedure text is split sentence by sentence to obtain a set of candidate function points;

[0012] The candidate function point set is deduplicated to obtain the function point set.

[0013] As an optional implementation of the method described in the first aspect, for each function point in the set of function points, the function of the function point is implemented by means of code and by calling a large model, respectively. The function implemented by the two methods is then tested using the test data of the function points to obtain test results, specifically including:

[0014] For each function point in the set of function points, generate code to implement the function of the function point, run the code to process the test data of the function point, and obtain the first test result;

[0015] For each function point in the set of function points, the test data of the function point is processed using a large test model based on the function of the function point to obtain a second test result.

[0016] Specifically, for each function point in the set of function points, code is generated to implement the function of that function point, including:

[0017] Based on the functions of the aforementioned functional points, prompt words are constructed;

[0018] The prompt words are input to generate a large model, which yields the code used to implement the function points.

[0019] Specifically, based on the test results, the applicability indicators of the two implementation methods for the functional points are determined, including:

[0020] Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by means of code is determined;

[0021] Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

[0022] As an optional implementation of the method described in the first aspect, a functional description text for each functional point is generated. The functional description texts and the applicability indicators of the two implementation methods for the functional points are used as training samples to fine-tune the target large model, resulting in the evaluation model. Specifically, this includes:

[0023] The functional description text of the functional point, the two implementation methods, and the applicability index of the functional point in the two implementation methods are input into the target large model to fine-tune the target large model.

[0024] As an optional implementation of the method described in the first aspect, a functional description text for each functional point is generated. The functional description texts and the applicability indicators of the two implementation methods for the functional points are used as training samples to fine-tune the target large model, resulting in the evaluation model. Specifically, this includes:

[0025] The functional description text of the functional point and the two implementation methods are input into the target large model, which drives the target large model to evaluate the applicability of the two implementation methods to the functional point.

[0026] Based on the evaluation results of the target large model and the applicability indicators of the functional points in the two implementation methods, an incentive signal is constructed;

[0027] The excitation signal is fed back to the target large model to fine-tune the target large model.

[0028] Secondly, a data processing method is provided, including:

[0029] Obtain the standard operating procedure text to be processed, and break the standard operating procedure text to be processed into function points;

[0030] Construct the functional description text for the aforementioned functional points;

[0031] The functional description text of the functional point is input into the evaluation model, which then evaluates the applicability of the functional point implemented through code and the applicability implemented by calling a large model. The evaluation model is pre-trained using the evaluation model training method described above.

[0032] Based on the evaluation results output by the evaluation model, the implementation method of the function point is determined, and the function point is converted into programmatic business logic based on the implementation method.

[0033] Thirdly, an evaluation model training device is provided, including:

[0034] The first data acquisition module is used to acquire the standard operating procedure text;

[0035] The first function point decomposition module is used to construct a set of function points based on the standard operating procedure text;

[0036] The second data acquisition module is used to collect test data for each function point in the set of function points;

[0037] The testing module is used to implement the function of each function in the set of function points by means of code and by calling the large model, and to test the function implemented by the two methods using the test data of the function points to obtain test results.

[0038] The indicator generation module is used to determine the applicability indicators of the two implementation methods for the functional point based on the test results.

[0039] The training module is used to generate functional description text for each functional point, and uses the functional description text and the applicability index of the functional point in the two implementation methods as training samples to fine-tune the target large model and obtain the evaluation model.

[0040] As an optional implementation of the device described in the third aspect, the first function point decomposition module is specifically used for:

[0041] The standard operating procedure text is split sentence by sentence to obtain a set of candidate function points;

[0042] The candidate function point set is deduplicated to obtain the function point set.

[0043] As an optional implementation of the apparatus described in the third aspect, the test module is specifically used for:

[0044] For each function point in the set of function points, generate code to implement the function of the function point, run the code to process the test data of the function point, and obtain the first test result;

[0045] For each function point in the set of function points, the test data of the function point is processed using a large test model based on the function of the function point to obtain a second test result.

[0046] Specifically, the test module is used for:

[0047] Based on the functions of the aforementioned functional points, prompt words are constructed;

[0048] The prompt words are input to generate a large model, which yields the code used to implement the function points.

[0049] Specifically, the indicator generation module is used for:

[0050] Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by means of code is determined;

[0051] Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

[0052] As an optional implementation of the apparatus described in the third aspect, the training module is specifically used for:

[0053] The functional description text of the functional point, the two implementation methods, and the applicability index of the functional point in the two implementation methods are input into the target large model to fine-tune the target large model.

[0054] As an optional implementation of the apparatus described in the third aspect, the training module is specifically used for:

[0055] The functional description text of the functional point and the two implementation methods are input into the target large model, which drives the target large model to evaluate the applicability of the two implementation methods to the functional point.

[0056] Based on the evaluation results of the target large model and the applicability indicators of the functional points in the two implementation methods, an incentive signal is constructed;

[0057] The excitation signal is fed back to the target large model to fine-tune the target large model.

[0058] Fourthly, a data processing apparatus is provided, comprising:

[0059] The third data acquisition module is used to acquire the text of the standard operating procedure to be processed.

[0060] The second function point decomposition module is used to decompose the standard operating procedure text to be processed into function points;

[0061] The evaluation module is used to construct the functional description text of the functional point, input the functional description text of the functional point into the evaluation model, and drive the evaluation model to evaluate the applicability of the functional point implemented by code and the applicability implemented by calling a large model respectively; the evaluation model is pre-trained using the evaluation model training method described above;

[0062] The processing module determines the implementation method of the function point based on the evaluation results output by the evaluation model, and converts the function point into programmatic business logic based on the implementation method.

[0063] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when the computer program is run on an electronic device, causes the electronic device to perform the above-described evaluation model training method, or to perform the above-described data processing method.

[0064] Sixthly, an electronic device is provided, comprising:

[0065] At least one memory for storing programs;

[0066] At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the evaluation model training method described above, or to execute the data processing method described above.

[0067] The beneficial effects of the evaluation model training method described in the embodiments of this specification are as follows:

[0068] This method breaks down standard operating procedure text into functional points, then implements each functional point both through code and by calling a large model. Test data is then used to evaluate these two implementation methods, and the suitability metrics for each method are determined based on the test results. Using this suitability metric data, an evaluation model is trained that can quantitatively determine whether a functional point is better suited for code implementation or implementation using a large model, based on the functional point description.

[0069] When breaking down new standard operating procedure (SOP) texts into tasks, an evaluation model can be used to automatically evaluate the implementation methods of functional points. This allows for the automatic breakdown of the new SOP texts based on the evaluation results of the evaluation model, and the breakdown effect is more reliable.

[0070] The evaluation model training device, data processing method, and apparatus described in the embodiments of this specification also have the above-mentioned beneficial effects. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 A schematic flowchart of an evaluation model training method described in an embodiment of this specification is shown as an example.

[0073] Figure 2A schematic flowchart of a data processing method described in an embodiment of this specification is shown as an example.

[0074] Figure 3 A schematic diagram of the structure of an evaluation model training device described in an embodiment of this specification is shown as an example.

[0075] Figure 4 A schematic diagram of the structure of a data processing apparatus according to an embodiment of this specification is shown as an example.

[0076] Figure 5 An exemplary schematic diagram of the structure of an electronic device provided in an embodiment of this specification is shown. Detailed Implementation

[0077] First, it should be noted that the terminology used in the embodiments of this invention is for the purpose of describing specific embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0078] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of the embodiments. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0079] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0080] This specification provides one or more embodiments of an evaluation model training method, data processing method, and apparatus.

[0081] The evaluation model training method, data processing method, and apparatus described in one or more embodiments of this specification will be further described in detail below with reference to the accompanying drawings and specific embodiments. However, this detailed description does not constitute a limitation on the embodiments of this specification.

[0082] Please refer to Figure 1 , Figure 1 This diagram illustrates a process flow for training an evaluation model, as proposed in one or more embodiments of this specification. The method aims to train an evaluation model capable of quantitatively determining whether a function is better suited for code implementation or a large-scale model implementation, based on a function description.

[0083] It should be noted that this evaluation model training method can be implemented using the evaluation model training apparatus described in one or more embodiments of this specification, but is not limited to this apparatus. For example... Figure 1 As shown, the method includes steps S100 to S106.

[0084] S100: Construct a set of function points based on pre-collected standard operating procedure texts, and collect test data for each function point in the set of function points.

[0085] Standard Operating Procedures (SOPs) are documents that describe in detail a task, its workflow, and steps. They are standardized process guidelines for internal organization operations, typically containing detailed procedures for a specific business, product, or service, for reference and adherence by internal personnel. SOPs primarily include standardized processes, clearly defined responsibilities and authorities, emergency response measures, quality standards and assessments, and document management.

[0086] The functional points mentioned in this step refer to the results of granularly decomposing the standard operating procedure (SOP) text. It should be noted that the degree of granularity in decomposing the SOP text can be set according to requirements, and this embodiment does not impose any limitations on this. For example, the aforementioned functional points can be process nodes, input / output nodes, decision points, etc.

[0087] In some implementations, the standard operating procedure text can be split sentence by sentence to obtain a set of candidate function points, that is, each sentence in the standard operating procedure text can be regarded as a function point.

[0088] Considering that the candidate function points may overlap, the set of candidate function points can be deduplicated to obtain the function point set.

[0089] For each feature in the feature set, test data is collected. This test data consists of datasets used to verify the correctness of the feature's logic and the stability of its performance. For example, if a feature's function is to filter emails containing fraudulent information, then the test data for that feature would be multiple emails with manually labeled content indicating whether they are fraudulent.

[0090] S102: For each function point in the function point set, implement the function point's functionality using both code-based and large-scale model-based methods, and use the function point's test data to test the functionality implemented by both methods and obtain the test results.

[0091] Formal programming languages ​​(such as Python and Java) define deterministic task decomposition, steps, control flow (sequential, parallel, conditional), data flow, and error handling logic, making them suitable for describing the skeleton of precisely executable business logic. Natural language instructions, on the other hand, offer more flexible expression and richer semantic meaning, making them suitable for describing uncertain or difficult-to-predefine parts of business logic, primarily used for processing large models.

[0092] To test whether each feature is better suited to the code-based approach or the approach of calling a large model, the same test data can be used to test both implementation methods separately:

[0093] For each function in the function point set, generate code to implement that function. Run the code to process the test data for that function and obtain the first test result. Specifically, randomly select some large models, construct prompts based on the function's functionality, input the prompts into the large model, and obtain the code to implement the function's functionality.

[0094] For each function point in the function point set, the test data of that function point is processed using the large test model based on the function of that function point to obtain the second test result.

[0095] S104: Based on the test results, determine the applicability indicators of the two implementation methods for the function points.

[0096] The applicability metrics mentioned above can be set according to needs. For example, recall rate, precision, and other metrics can be selected as applicability metrics. The selection and number of applicability metrics can be set according to needs, and this embodiment does not impose any restrictions on this.

[0097] Specifically, the suitability indicators for implementing a function point through code can be determined based on the first test results of the function point and the standard reference results of the test data of the function point.

[0098] Based on the second test results of the function points and the standard reference results of the test data of the function points, it is determined that the applicability index of the function points can be achieved by calling the large model.

[0099] The aforementioned standard reference results can be manually annotated specific information used to evaluate whether the first and second test results are correct.

[0100] S106: Generate the functional description text for each functional point, and use the functional description text and the applicability index of the two implementation methods mentioned above as training samples to fine-tune the target large model and obtain the evaluation model.

[0101] The target large model here can be a randomly selected large language model.

[0102] When fine-tuning the target large model, the functional description text of the functional points, two implementation methods (implemented through code and implemented through the large model), and the applicability index of the functional points in these two implementation methods can be input into the target large model to fine-tune the target large model.

[0103] Alternatively, reinforcement learning can be used to fine-tune the target large model. Specifically, the functional description text and two implementation methods (implemented via code and implemented via the large model) can be input into the target large model. Prompt words can then drive the target large model to evaluate the applicability of each implementation method for the functional point. The evaluation metric must be consistent with the applicability metric mentioned above; that is, if the applicability metric uses accuracy, then the evaluation metric should also use accuracy. Based on the evaluation results of the target large model and the applicability metric of the functional point in both implementation methods, an incentive signal is constructed. This incentive signal is then fed back to the target large model for fine-tuning.

[0104] The above describes an evaluation model training method as illustrated in one or more embodiments of this specification. This method decomposes the standard operating procedure text into functional points, then implements each functional point both through code and by calling a large model. Test data is then used to test these two implementation methods, and the suitability indicators of each method for the functional point are determined based on the test results. Using these suitability indicator data, an evaluation model is trained that can quantitatively determine whether a functional point is better implemented using code or a large model, based on the functional point description.

[0105] Corresponding to the above-described evaluation model training method, one or more embodiments of this specification propose a data processing method. For example... Figure 2 As shown, the method includes steps S200 to S206.

[0106] S200: Obtain the standard operating procedure text to be processed and break it down into function points.

[0107] S202: Functional description text for building functional points.

[0108] S204: Input the functional description text of the function point into the evaluation model, and drive the evaluation model to evaluate the applicability of the function point implemented by code and the applicability implemented by calling the large model respectively.

[0109] S206: Based on the evaluation results output by the evaluation model, determine the implementation method of the function points, and convert the function points into programmatic business logic based on the implementation method.

[0110] The evaluation model used in the above data processing method is pre-trained using the evaluation model training method described above. The training process and working principle of the evaluation model have been described in detail in the evaluation model training method section above, and will not be repeated here.

[0111] This data processing method can automatically evaluate the implementation of functional points by using an evaluation model when breaking down new standard operating procedure texts into tasks. This allows for the automatic breakdown of new standard operating procedure texts based on the evaluation results of the evaluation model, and the breakdown effect is more reliable.

[0112] Corresponding to the above-described evaluation model training method, one or more embodiments of this specification propose an evaluation model training device. For example... Figure 3 As shown, the evaluation model training device includes:

[0113] The first data acquisition module 301 is used to acquire the standard operating procedure text.

[0114] The first function point decomposition module 302 is used to construct a set of function points based on the standard operating procedure text.

[0115] The second data acquisition module 303 is used to collect test data for each function point in the function point set.

[0116] Test module 304 is used to implement the functionality of each functionality in the functionality set by either code or by calling the large model, and to test the functionality implemented by the two methods using the test data of the functionality, and obtain the test results.

[0117] The indicator generation module 305 is used to determine the applicability indicators of the two implementation methods for the function points based on the test results.

[0118] Training module 306 is used to generate functional description text for each functional point. The functional description text and the applicability index of the functional point in two implementation methods are used as training samples to fine-tune the target large model and obtain the evaluation model.

[0119] Optionally, the first function point decomposition module 302 described above is specifically used to: decompose the standard operating procedure text sentence by sentence to obtain a set of candidate function points; and perform deduplication processing on the set of candidate function points to obtain a set of function points.

[0120] Optionally, the aforementioned test module 304 is specifically used for:

[0121] For each function in the function set, generate code to implement the function's functionality, run the code to process the function's test data, and obtain the first test result;

[0122] For each function point in the function point set, the test data of the function point is processed based on the function of the function point using the large test model to obtain the second test result.

[0123] Specifically, the aforementioned test module 304 is also used to construct prompt words based on the function of the function point; input the prompt words to generate a large model, and obtain the code used to implement the function of the function point.

[0124] Specifically, the aforementioned indicator generation module 305 is used for:

[0125] Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by code is determined;

[0126] Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

[0127] Optionally, the training module 306 described above can be used to fine-tune the target large model in the following ways:

[0128] Input the functional description text of the function point, two implementation methods, and the applicability index of the function point in the two implementation methods into the target large model to fine-tune the target large model.

[0129] Alternatively, the training module 306 described above can also be used to fine-tune the target large model in the following way:

[0130] Input the functional description text of the function point and the two implementation methods into the target large model, and drive the target large model to evaluate the applicability of the two implementation methods to the function point respectively;

[0131] Based on the evaluation results of the target model and the applicability indicators of this function in the two implementation methods, an incentive signal is constructed.

[0132] The excitation signal is fed back to the target large model to fine-tune the target large model.

[0133] Below, with Figure 3The implementation principle of this device is illustrated using the evaluation model training device shown as an example. Taking a module as a software functional unit as an example, the first data acquisition module 301 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, the first data acquisition module 301 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically proximate data centers. Typically, a region may include multiple AZs.

[0134] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0135] As an example of a hardware functional unit, the first data acquisition module 301 may include at least one computing device, such as a server. Alternatively, the first data acquisition module 301 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0136] The multiple computing devices included in the first data acquisition module 301 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the first data acquisition module 301 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the first data acquisition module 301 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0137] In other embodiments, the first data acquisition module 301 can be used to execute any step in the above-described evaluation model training method; the first function point decomposition module 302 can be used to execute any step in the above-described evaluation model training method; the second data acquisition module 303 can be used to execute any step in the above-described evaluation model training method; the testing module 304 can be used to execute any step in the above-described evaluation model training method; the index generation module 305 can be used to execute any step in the above-described evaluation model training method; and the training module 306 can be used to execute any step in the above-described evaluation model training method. The steps implemented by the first data acquisition module 301, the first function point decomposition module 302, the second data acquisition module 303, the testing module 304, the index generation module 305, and the training module 306 can be specified as needed. The evaluation model training device achieves all its functions by implementing different steps in the above-described evaluation model training method through the first data acquisition module 301, the first function point decomposition module 302, the second data acquisition module 303, the testing module 304, the index generation module 305, and the training module 306 respectively.

[0138] In this implementation, the device can also be applied to computing devices such as computers and servers, or to a cluster of computing devices including at least one computing device, in order to implement specific evaluation model training.

[0139] Corresponding to the above-described evaluation model training method, one or more embodiments of this specification propose a data processing apparatus. For example... Figure 4 As shown, the data processing device includes:

[0140] The third data acquisition module 401 is used to acquire the standard operating procedure text to be processed.

[0141] The second function point decomposition module 402 is used to decompose the standard operating procedure text to be processed into function points.

[0142] Evaluation module 403 is used to construct the functional description text of the functional points. The functional description text of the functional points is input into the evaluation model, which drives the evaluation model to evaluate the applicability of the functional points implemented by code and the applicability implemented by calling a large model. The evaluation model is pre-trained using the evaluation model training method described above.

[0143] Processing module 404 determines the implementation method of the function point based on the evaluation results output by the evaluation model, and converts the function point into programmatic business logic based on the implementation method.

[0144] Below, with Figure 4 The implementation principle of the device is illustrated using the data processing apparatus shown as an example. Taking a module as a software functional unit as an example, the third data acquisition module 401 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, the third data acquisition module 401 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0145] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0146] As an example of a hardware functional unit, the third data acquisition module 401 may include at least one computing device, such as a server. Alternatively, the third data acquisition module 401 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0147] The multiple computing devices included in the third data acquisition module 401 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the third data acquisition module 401 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the third data acquisition module 401 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0148] In other embodiments, the third data acquisition module 401 can be used to execute any step in the above-described data processing method, the second function point decomposition module 402 can be used to execute any step in the above-described data processing method, the evaluation module 403 can be used to execute any step in the above-described data processing method, and the processing module 404 can be used to execute any step in the above-described data processing method. The steps implemented by the third data acquisition module 401, the second function point decomposition module 402, the evaluation module 403, and the processing module 404 can be specified as needed. By implementing different steps in the above-described data processing method through the third data acquisition module 401, the second function point decomposition module 402, the evaluation module 403, and the processing module 404, all the functions of the above-described data processing device can be realized.

[0149] In this implementation, the device can also be applied to computing devices such as computers and servers, or to a cluster of computing devices including at least one computing device, to implement a specific data processing method.

[0150] One or more embodiments described in this specification also provide an electronic device. Please refer to... Figure 5The electronic device includes a bus 501, a processor 502, a memory 503, and a communication interface 504. The processor 502, memory 503, and communication interface 504 communicate via the bus 501. This electronic device can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the electronic device.

[0151] Bus 501 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus 501 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 501 may include a path for transmitting information between various components of an electronic device (e.g., processor 502, memory 503, and communication interface 504).

[0152] Processor 502 may include any one or more processors such as CPU, graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0153] Memory 503 may include volatile memory, such as random access memory (RAM). Memory 503 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0154] The memory 503 stores executable program code, and the processor 502 executes the executable program code to implement the aforementioned evaluation model training method, or to implement the aforementioned data processing method.

[0155] Communication interface 504 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between electronic devices and other devices or communication networks.

[0156] One or more embodiments of this specification provide a computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the evaluation model training method described above, or to perform the data processing method described above.

[0157] The computer-readable storage medium can be any available medium that an electronic device can store, or a data storage device such as a data center that contains one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives). The computer-readable storage medium includes instructions that instruct the electronic device to perform the aforementioned evaluation model training method, or to perform the aforementioned data processing method.

[0158] It is understood that the structures illustrated in the embodiments of this specification do not constitute a specific limitation on the system of the embodiments of this specification. In other embodiments of the specification, the above system may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0159] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0160] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0161] It should be noted that the above examples are merely specific embodiments of the present invention, and the present invention is obviously not limited to the above embodiments, with many similar variations. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in this invention should fall within the protection scope of this invention.

Claims

1. A method for evaluating model training, comprising: A set of function points is constructed based on pre-collected standard operating procedure texts, and test data is collected for each function point in the set of function points; For each function point in the set of function points, the function point is implemented by means of code and by calling the large model, respectively. The function implemented by means of code and by calling the large model is tested using the test data of the function points, and the test results are obtained. Based on the test results, determine the applicability indicators of the code method and the method of calling the large model for the functional point; Generate a functional description text for each functional point, and use the functional description text, the code method, and the method of calling the large model as applicability indicators for the functional point as training samples to fine-tune the target large model and obtain the evaluation model.

2. The method according to claim 1, wherein constructing a set of function points based on pre-collected standard operating procedure text specifically includes: The standard operating procedure text is split sentence by sentence to obtain a set of candidate function points; The candidate function point set is deduplicated to obtain the function point set.

3. The method according to claim 1, wherein for each function point in the set of function points, the function of the function point is implemented by means of code and by calling a large model, respectively, and the function implemented by means of code and by calling a large model is tested using the test data of the function points, and test results are obtained, specifically including: For each function point in the set of function points, generate code to implement the function of the function point, run the code to process the test data of the function point, and obtain the first test result; For each function point in the set of function points, the test data of the function point is processed using a large test model based on the function of the function point to obtain a second test result.

4. The method according to claim 3, for each function point in the set of function points, generating code to implement the function of the function point, specifically includes: Based on the functions of the aforementioned functional points, prompt words are constructed; The prompt words are input to generate a large model, which yields the code used to implement the function points.

5. The method according to claim 3, based on the test results, determining the applicability index of the code method and the method of calling the large model for the functional point, specifically including: Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by code is determined; Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

6. According to claim 1, the method involves generating a functional description text for each functional point, using the functional description text of each functional point, along with the applicability index of the code method and the method of calling the large model for each functional point, as training samples to fine-tune the target large model and obtain the evaluation model, specifically including: The functional description text of the functional point, the code method, the method of calling the large model, and the applicability index of the functional point on the code method and the method of calling the large model are input into the target large model to fine-tune the target large model.

7. The method according to claim 1, generating functional description text for each functional point, using the functional description text of the functional point, along with the applicability index of the code method and the method of calling the large model for the functional point as training samples, fine-tuning the target large model to obtain the evaluation model, specifically including: The function description text of the function point, the code method, and the method of calling the large model are input into the target large model, and the target large model is driven to evaluate the applicability of the code method and the method of calling the large model to the function point respectively; Based on the evaluation results of the target large model and the applicability index of the functional points in the code method and the method of calling the large model, an incentive signal is constructed; The excitation signal is fed back to the target large model to fine-tune the target large model.

8. A data processing method, comprising: Obtain the standard operating procedure text to be processed, and break the standard operating procedure text to be processed into function points; Construct the functional description text for the aforementioned functional points; Input the functional description text of the functional point into the evaluation model, and drive the evaluation model to evaluate the applicability of the functional point implemented by code and the applicability implemented by calling a large model respectively; The evaluation model is pre-trained using the method described in any one of claims 1 to 7; Based on the evaluation results output by the evaluation model, the implementation method of the function point is determined, and the function point is converted into programmatic business logic based on the implementation method.

9. An evaluation model training device, comprising: The first data acquisition module is used to acquire the standard operating procedure text; The first function point decomposition module is used to construct a set of function points based on the standard operating procedure text; The second data acquisition module is used to collect test data for each function point in the set of function points; The testing module is used to implement the function of each function in the set of function points by means of code and by calling the large model, respectively, and to test the function implemented by means of code and by calling the large model using the test data of the function points, and to obtain test results. The indicator generation module is used to determine the applicability indicators of the coding method and the method of calling the large model for the functional point based on the test results; The training module is used to generate a functional description text for each functional point, and uses the functional description text and the applicability index of the functional point on the code method and the method of calling the large model as training samples to fine-tune the target large model and obtain the evaluation model.

10. The apparatus according to claim 9, wherein the first function point decomposition module is specifically used for: The standard operating procedure text is split sentence by sentence to obtain a set of candidate function points; The candidate function point set is deduplicated to obtain the function point set.

11. The apparatus according to claim 9, wherein the testing module is specifically used for: For each function point in the set of function points, generate code to implement the function of the function point, run the code to process the test data of the function point, and obtain the first test result; For each function point in the set of function points, the test data of the function point is processed using a large test model based on the function of the function point to obtain a second test result.

12. The apparatus according to claim 11, wherein the testing module is specifically used for: Based on the functions of the aforementioned functional points, prompt words are constructed; The prompt words are input to generate a large model, which yields the code used to implement the function points.

13. The apparatus according to claim 11, wherein the index generation module is specifically used for: Based on the first test results of the function point and the standard reference results of the test data of the function point, the applicability index of implementing the function point by code is determined; Based on the second test results of the function point and the standard reference results of the test data of the function point, it is determined that the applicability index of the function point can be achieved by calling the large model.

14. The apparatus according to claim 9, wherein the training module is specifically used for: The functional description text of the functional point, the code method, the method of calling the large model, and the applicability index of the functional point on the code method and the method of calling the large model are input into the target large model to fine-tune the target large model.

15. The apparatus according to claim 9, wherein the training module is specifically used for: The functional description text of the functional point, the code method, and the method of calling the large model are input into the target large model, which drives the target large model to evaluate the applicability of the code method and the method of calling the large model to the functional point respectively; Based on the evaluation results of the target large model and the applicability index of the functional points in the code method and the method of calling the large model, an incentive signal is constructed; The excitation signal is fed back to the target large model to fine-tune the target large model.

16. A data processing apparatus, comprising: The third data acquisition module is used to acquire the text of the standard operating procedure to be processed. The second function point decomposition module is used to decompose the standard operating procedure text to be processed into function points; The evaluation module is used to construct the functional description text of the functional point, input the functional description text of the functional point into the evaluation model, and drive the evaluation model to evaluate the applicability of the functional point implemented by code and the applicability implemented by calling a large model respectively. The evaluation model is pre-trained using the method described in any one of claims 1 to 7; The processing module determines the implementation method of the function point based on the evaluation results output by the evaluation model, and converts the function point into programmatic business logic based on the implementation method.

17. A computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the method as claimed in any one of claims 1 to 7, or to perform the method as claimed in claim 8.

18. An electronic device comprising: At least one memory for storing programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as claimed in any one of claims 1 to 7, or to perform the method as claimed in claim 8.

Images