Data processing method and device

Abstract

The invention discloses a data processing method and device, relates to the field of science and technology finance, and is used for solving the problem that many data with poor quality and high repetition rate exist in sample data. The method comprises the steps that multiple pieces of data are obtained, each piece of data in the multiple pieces of data comprises instruction data and output result data, the output result data are obtained by analyzing the instruction data through a large language model, and for each piece of data in the multiple pieces of data, an initial quality value of the data is determined according to a preset data index rule; and determining the similarity between the data and each piece of other data in the multiple pieces of data, correcting the initial quality value of the data by using the similarity between the data and each piece of other data in the multiple pieces of data to obtain a final quality value of the data, and determining target data in the multiple pieces of data according to the final quality value of the multiple pieces of data. Through the scheme, data with diversity and high quality can be screened.

Description

Technical Field

[0001] The present invention relates to the field of Fintech, and in particular to a data processing method and device. Background Art

[0002] With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually transforming to Fintech. In the field of Fintech, code development is usually involved. Developing various financial analysis and transaction software or applications through code can facilitate the staff in the field of Fintech to more reasonably screen financial data and improve the speed and accuracy of processing financial transactions.

[0003] In the process of code development, large language models can be used to effectively assist various development links, such as using large language models to detect the accuracy of the developed code. Since different developers have their own development styles, it is necessary to make necessary fine-tuning of the large language model to adapt to these differences. In the process of fine-tuning the large language model, it is necessary to provide effective high-quality data to improve the quality of the large language model, so as to reduce the cost of fine-tuning the large language model, enable the large language model to learn key information faster, reduce the number and time of iterations required for training, and reduce the time for manual screening and cleaning of data, thereby reducing labor costs. Therefore, in order to improve the generalization and innovation of the large language model, it is necessary to use diverse and high-quality training data as much as possible in the process of training the model.

[0004] However, there are many data with poor quality and high duplication in the current sample data. Therefore, how to screen out diverse and high-quality data from a large amount of sample data is a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present invention provide a data processing method and device for screening data with diversity and high quality.

[0006] In a first aspect, an embodiment of the present invention provides a data processing method, the method comprising: acquiring multiple data, each of the multiple data comprising instruction data and output result data, the output result data being obtained by analyzing the instruction data using a large language model, for each of the multiple data, determining an initial quality value of the data according to a preset data indicator rule, the initial quality value of the data being used to indicate the quality value of the output result data in the data, the higher the degree of correspondence between the output result data in the data and the instruction data in the data, the greater the quality value of the output result data in the data, for each of the multiple data, determining the similarity between the data and each other data in the multiple data, for each of the multiple data, using the similarity between the data and each other data in the multiple data to correct the initial quality value of the data, to obtain a final quality value of the data, the final quality value being used to indicate the diversity of the instruction data of the data in the instruction data of all the data and the quality of the output result data of the data, and determining target data in the multiple data according to the final quality values ​​of the multiple data; the target data is data whose final quality value is greater than a first threshold, or the target data is a set number of data with the largest final quality value.

[0007] Through the above scheme, the initial quality value of the data is determined according to the set data indicator rules, high-quality data can be screened out, and the initial quality value of the data is corrected using similarity, which can screen out data with high similarity. In this way, it can be ensured that the target data finally determined is diverse and high-quality data.

[0008] Optionally, for each of the multiple data, an initial quality value corresponding to the data is determined according to a preset data indicator rule, including: using a preset first wording, inputting the preset data indicator rule and the data into a preset plurality of large language models, obtaining a plurality of reference quality values ​​output by the preset plurality of large language models, the preset first wording is used to instruct the large language model to output a reference quality value, obtaining a first weight corresponding to each of the preset plurality of large language models, and taking the sum of the products of the first weight corresponding to each of the preset plurality of large language models and the reference quality value as the initial quality value of the data.

[0009] Through the above scheme, by integrating the evaluation results of multiple models, the deviation problem caused by single model evaluation can be solved, making the initial quality value more accurate, and allowing the weights of different models to be adjusted according to actual conditions, thereby increasing the flexibility of evaluation.

[0010] Optionally, after determining the initial quality value of the data according to preset data indicator rules and before determining the similarity between the data and each other data in multiple data, it also includes: using a preset second speech, sending the preset data indicator rules and the data as target information to the large language model, and obtaining the probability value output by the large language model, the preset second speech is used to indicate the probability value that the data in the target information output by the large language model meets the preset indicator rules in the target information, and using the product of the probability value and the initial quality value to update the initial quality value.

[0011] Through the above scheme, determining the probability that the data meets the preset data indicator rules can make up for the problem of different confidence levels in the data scoring process of the large language model, that is, taking into account the confidence level of the large language model score, making the initial quality value more accurate.

[0012] Optionally, for each piece of data in the plurality of pieces of data, the initial quality value of the data is corrected using the similarity between the data and each other piece of data in the plurality of pieces of data to obtain the final quality value of the data, including: obtaining the initial quality value of each other piece of data, and dividing the similarity between the data and each other piece of data by the initial quality value of each other piece of data to obtain the correction ratio of each other piece of data to the data, calculating the sum of the correction ratios of each other piece of data to the data, and taking the product of the sum and the initial quality value as the final quality value of the data. In this way, by calculating the correction ratio through the similarity between the data and other data, and thereby correcting the initial quality value, data with high quality and low similarity can be screened out.

[0013] Optionally, the preset data indicator rules include one or more of the following: code comments meet specification requirements, the actual completed answer is consistent with the requirements of the instruction data, the generated answer is correct and can be successfully executed, the generated answer is fluent and there is no ambiguity, the format of the generated answer meets the requirements, and the format of the generated answer can fully explain the answer. In this way, data that meets the preset data indicator rules can be screened out.

[0014] Optionally, multiple data belong to N scenes, where N is an integer greater than or equal to 2. After obtaining the final quality value of the data, it also includes: determining the target scene to which the data belongs, calculating the sum of the initial quality values ​​of each data contained in the target scene, dividing the sum by the total amount of data in the target scene to obtain an average quality value corresponding to the target scene, dividing the initial quality value of the data by the average quality value to obtain a second weight of the data, and taking the product of the second weight and the final quality value as the corrected final quality value.

[0015] Through the above scheme, data in different scenarios can be mapped to one dimension for quality measurement and comparison. In this way, target data with multiple scenarios can be screened out.

[0016] Optionally, the N scenarios include one or more of the following: a python code generation scenario, a java code generation scenario, a python risk scanning scenario, and a java risk scanning scenario.

[0017] Optionally, for each of the multiple data, before using the similarity between the data and each other data in the multiple data to correct the initial quality value of the data, the method further includes: deleting other data with a large similarity according to the similarity between the data and each other data. In this way, subsequent calculation steps can be simplified.

[0018] In a second aspect, an embodiment of the present invention provides a data processing device, the data processing device comprising:

[0019] The acquisition module is used to acquire multiple pieces of data, each of which includes instruction data and output result data, and the output result data is obtained by analyzing the instruction data using a large language model.

[0020] an initial quality value determination module, configured to determine, for each of the plurality of data, an initial quality value of the data according to a preset data indicator rule, wherein the initial quality value of the data is used to indicate a quality value of output result data in the data, and the higher the degree of correspondence between the output result data in the data and the instruction data in the data, the greater the quality value of the output result data in the data;

[0021] A similarity determination module, configured to determine, for each piece of data in the plurality of pieces of data, a similarity between the data and each other piece of data in the plurality of pieces of data;

[0022] A correction module is used to correct the initial quality value of each of the multiple data using the similarity between the data and each other data in the multiple data to obtain a final quality value of the data, wherein the final quality value is used to indicate the diversity of the instruction data of the data in the instruction data of all data and the quality of the output result data of the data.

[0023] The target data determination module is used to determine the target data among the multiple data according to the final quality values ​​of the multiple data; the target data is the data whose final quality value is greater than a first threshold, or the target data is a set number of data with the largest final quality value.

[0024] In one possible implementation, the initial quality value determination module is specifically used to: use a preset first wording, input preset data indicator rules and data into multiple preset large language models, obtain multiple reference quality values ​​output by the preset multiple large language models, the preset first wording is used to instruct the large language model to output the reference quality value, obtain the first weight corresponding to each of the preset multiple large language models, and use the sum of the products of the first weight corresponding to each of the preset multiple large language models and the reference quality value as the initial quality value of the data.

[0025] In a possible implementation, it also includes an initial quality value updating module, which is used to use a preset second speech to send preset data indicator rules and data as target information to the large language model to obtain the probability value of the large language model output. The preset second speech is used to indicate the probability value that the data in the target information output by the large language model meets the preset indicator rules in the target information, and the initial quality value is updated using the product of the probability value and the initial quality value.

[0026] In one possible implementation, the correction module is specifically used to: obtain the initial quality value of each other data, and divide the similarity between the data and each other data by the initial quality value of each other data to obtain the correction ratio of each other data to the data, calculate the sum of the correction ratios of each other data to the data, and use the product of the sum and the initial quality value as the final quality value of the data.

[0027] In one possible implementation, the preset data indicator rules include one or more of the following: the code comments meet the specification requirements, the actual completed answers are consistent with the requirements of the instruction data, the generated answers are correct and can be successfully executed, the generated answers are fluent and unambiguous, the format of the generated answers meets the requirements, and the format of the generated answers can fully explain the answers.

[0028] In one possible implementation, multiple data belong to N scenes, N is an integer greater than or equal to 2. After obtaining the final quality value of the data, the target data determination module is also used to: determine the target scene to which the data belongs, calculate the sum of the initial quality values ​​of each data contained in the target scene, divide the sum by the total amount of data in the target scene to obtain the average quality value corresponding to the target scene, divide the initial quality value of the data by the average quality value to obtain a second weight of the data, and use the product of the second weight and the final quality value as the corrected final quality value.

[0029] In a possible implementation, the N scenarios include one or more of the following: a python code generation scenario, a java code generation scenario, a python risk scanning scenario, and a java risk scanning scenario.

[0030] In a possible implementation, a deletion module is further included, which is used to delete other data with a large similarity according to the similarity between the data and each other data.

[0031] In a third aspect, the present application also provides a data processing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in the various possible designs of the first aspect.

[0032] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program or instructions are stored. When the computer program or instructions are executed by a processor, the method described in the various possible designs of the first aspect is implemented.

[0033] In a fifth aspect, an embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0034] These implementations or other implementations of the present application will be more concise and understandable in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A schematic diagram of using a large language model to score data provided by an embodiment of the present invention;

[0037] Figure 2 A flowchart of a data processing method provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a process for determining initial quality values ​​corresponding to data under multiple large language models provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a process for determining a final quality value provided by an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of a process for determining target data according to a target scenario to which the data belongs provided by an embodiment of the present invention;

[0041] Figure 6A schematic diagram of a process for correcting an initial quality value using probability provided by an embodiment of the present invention;

[0042] Figure 7 A schematic diagram of using a large language model to determine whether data conforms to a preset data indicator rule provided by an embodiment of the present invention;

[0043] Figure 8 A schematic diagram of a data processing device provided by an embodiment of the present invention;

[0044] Fig. 9 A schematic diagram of another data processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and beneficial effects of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] Below, some terms in this application are explained. It should be noted that these explanations are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed in this application.

[0047] 1. Vector.

[0048] A vector is a mathematical concept used to represent a quantity with magnitude and direction. In the fields of machine learning and natural language processing, vectors can be used to represent data. For example, text data can be converted into high-dimensional vectors (such as 1024 dimensions).

[0049] 2. Cosine similarity.

[0050] Cosine similarity is used to measure the similarity between two vectors. The higher the cosine similarity, the more similar the two vectors are. Conversely, the lower the cosine similarity, the less similar the two vectors are.

[0051] 3. Large language model (LLM).

[0052] LLM is a natural language processing model based on deep learning technology. It can understand and generate human language through training with massive text data.

[0053] 4. Training data

[0054] The training data is used to train the large language model, so that the large language model has better generalization. The training data can be different according to different user needs.

[0055] In an embodiment of the present invention, training data is data related to code. In actual application scenarios, with the continuous development of business, the code development department has accumulated a lot of data, such as source code files, etc. The source code file includes code and annotation information corresponding to the code. If the code and the annotation information corresponding to the code are extracted from the source code file, the annotation information is used as the code instruction, the code corresponding to the annotation information is used as the output result corresponding to the code instruction, and the code instruction and the output result are used as training data. For example, the following are two examples of training data:

[0056] Training data 1:

[0057] Instructions: Please write a quick sort algorithm in Python

[0058] Output: Python implements the quick sort algorithm as follows: def quick_sort():....

[0059] Training data 2:

[0060] Instructions: Please check whether the provided code contains any risks. If so, please point out the risks and provide corresponding solutions. The provided code is: def load_data():.....

[0061] Output: The provided code does not check the variables. When the input file is empty, an exception will occur. It is recommended to modify it to def load_data():....

[0062] By training a large language model with many training data similar to the above, the large language model can learn relevant information. For example, when annotation information is input into the large language model, the large language model can output the corresponding code.

[0063] However, as described in the background technology, there are many data with poor quality and high repetition rate in the training data. Currently, the methods for screening the data in the training data mainly include the following method 1 and method 2.

[0064] Method 1: Evaluate the quality of data based on rule matching, including the following steps 1 to 3.

[0065] Step 1: Design preset rules regarding data quality.

[0066] For example, the following three preset rules:

[0067] Rule A1: There is a keyword in the text.

[0068] Rule A2: The text should be between 50 and 5000 words.

[0069] Rule A3: Function names should be in camelCase.

[0070] Step 2: Set the weight corresponding to each preset rule.

[0071] For example, set corresponding weights for the following three preset rules:

[0072] Rule A1: There is a keyword in the text, and the weight is 0.1.

[0073] Rule A2: If the number of words in a text is between 50 and 5000, the weight is 0.1.

[0074] Rule A3: Function names are named in camel case, with a weight of 0.5.

[0075] Step three: match the data with the preset rules, and calculate the evaluation result of the data based on the matching result and the weight corresponding to the matched preset rules.

[0076] For example, if the number of words in a piece of data is 100 and the function of the data is named in camel case, the rules matched by the data are rule 2 and rule 3. The weight corresponding to rule 2 is 0.1, and the weight corresponding to rule 3 is 0.5. Therefore, the evaluation result of the data is 0.6.

[0077] Although the above method 1 can evaluate the quality of data according to rule matching, it does not take into account the semantic differences, which can easily lead to errors in the evaluation results. For example, if two data have the same semantics but are expressed in completely different ways, the evaluation results of the two data obtained according to method 1 will be very different.

[0078] Based on this, considering the semantics of the data, the industry has proposed the following method 2.

[0079] Method 2: Evaluate data quality based on other large language models, including steps 1 to 4 below.

[0080] Step 1: Set one or more scoring rules.

[0081] For example, rules can be set based on code format specifications, code function implementation, answer fluency, whether the code can be run, and the degree of match between the answer content and the question, etc. Specifically:

[0082] B1. Code comments meet specification requirements.

[0083] B2. The actual completed answer is consistent with the instruction requirements.

[0084] B3. The generated answer is correct and can be successfully executed.

[0085] B4. The generated answers are fluent and unambiguous.

[0086] B5. The format of the generated answers meets the requirements and can fully explain the answers.

[0087] Step 2: Choose a suitable large language model.

[0088] The large language model here can be consistent with the large language model that needs to be fine-tuned, or it can be inconsistent. You can choose a model that is currently open source and has strong evaluation capabilities for text-type data, such as the qwen-72b model.

[0089] Step 3: Use the selected large language model to score the data.

[0090] Specifically, you can first formulate multiple scoring rules, and use the selected large language model to score the data. If the data meets one of the multiple scoring rules, the score of the data is +1, otherwise, no score is given. For example, please refer to Figure 1 An example of using a large language model to score data is shown. Figure 1 The above training data 2 is taken as an example, that is, the above training data 1 is scored using the large language model. Figure 1 The content is sent to the large language model, and the large language model will output

[0091] Answer in this format: {“score”:,”reason”:}.

[0092] Step 4: Analyze the output results of the big language to obtain the corresponding scoring results, filter the data according to the scoring results, and then select the data with high scores.

[0093] Using the above method 2, a large language model can be used to score data quality. However, this solution has the following three disadvantages:

[0094] Disadvantage 1: The filtered data are generally very similar, resulting in poor data diversity.

[0095] Disadvantage 2: The screened data scenario is single, and the data in different scenarios are uneven.

[0096] Real data has different scenarios, including but not limited to: python code generation scenario, java code generation scenario, python risk scanning scenario, java risk scanning scenario, etc. For example, in the above training data example, training data 1 corresponds to the python code generation scenario, and training data 2 corresponds to the python risk scanning scenario.

[0097] Since Method 2 does not set different preset rules according to different data scenarios in step 1, Method 2 will have deviations when evaluating data in different scenarios. For example, taking the evaluation of the python instruction generation scenario and the python risk scanning scenario as an example, in the above step 3, the python instruction generation is more inclined to meet the above rules 1, 2, 3, 4, and 5, while the python risk scanning scenario is more inclined to meet the above rules 2, 4, and 5, that is, the first and third rules are not a very critical evaluation criteria for the python risk scanning scenario. In other words, if the same set of standards is used to measure data in different scenarios, the final filtered data will be uneven, and it is very likely that all the data in the two scenarios of python code generation or java code generation that meet the above conditions will be filtered out, while all the data of python risk scanning or java risk scanning will be filtered out.

[0098] Disadvantage 3: It relies on the capabilities of a large language model and the scoring is inaccurate.

[0099] The second method is to use a large language model for scoring. However, the confidence level of the large language model for scoring each piece of data is different (confidence level refers to the degree of reliability of a prediction or estimation result in machine learning. For example, when the large language model is powerful, the result given is more in line with reality, and when the large language model is weak, the result given has a large deviation). For example, the following two examples:

[0100] Example 1: {“score”:5,”reason”:Based on the above information, the score of this data is 5}

[0101] Example 2: {“score”:5,”reason”: Since this data meets the above 5 requirements, we give it a score of 5. This score indicates that the data has high data quality.}

[0102] Examples 1 and 2 above are both the scoring outputs of a piece of data by the large language model. Obviously, their score values ​​are both 5 points, but in this process they do not reflect the confidence of this score, that is, they do not reflect the probability with which the current large language model outputs this series of results.

[0103] For example, if one string of code compiles 20% and another string of code compiles 80%, the big language model will give the two strings of code the same score, which is 0 (because neither string of code compiles 100%). For another example, if the data format is not standardized and the data format is very irregular, the big language model will give the same score of 0.

[0104] In summary, the two data screening methods provided by the industry cannot take into account both the quality and diversity of text data.

[0105] Based on this, the present invention provides a data processing method, which screens data with diversity and high quality by determining the similarity between each piece of data and other data.

[0106] The present application scheme is described in detail below in conjunction with the accompanying drawings.

[0107] See also Figure 2 , a flow chart of a data processing method is shown, which specifically includes the following steps:

[0108] Step 210: Acquire multiple pieces of data.

[0109] For example, multiple text information can be obtained from a database, or multiple text information can be collected from a document, or multiple training data similar to the above description can be obtained from a code repository, for example, 500,000 pieces of data, wherein each of the multiple pieces of data includes two parts: instruction data and output result data, and the output result data is obtained by analyzing the instruction data using a large language model, such as the training data 1 and training data 2 listed above.

[0110] Step 220: For each piece of data among the multiple pieces of data, determine an initial quality value corresponding to the data according to a preset data indicator rule.

[0111] The initial quality value of the data is used to indicate the quality value of the output result data in the data. The higher the correspondence between the output result data in the data and the instruction data in the data, the greater the quality value of the output result data in the data.

[0112] Before step 220, it is necessary to set the data indicator rules. Optionally, the set data indicator rules may include all or part of the rules shown in the above content, such as but not limited to rules A1 to A3 in the above method 1, or may also include rules B1 to B5 in the above method 2.

[0113] Specifically, the preset data indicator rules may include one or more of the following: the code comments meet the specification requirements, the actual completed answer is consistent with the requirements of the instruction data, the generated answer is correct and can be successfully executed, the generated answer is fluent and there is no ambiguity, the format of the generated answer meets the requirements, and the format of the generated answer can fully explain the answer. In this way, data that meets the preset data indicator rules can be screened out.

[0114] In step 220 , if 500,000 pieces of data are obtained in step 210 , 500,000 initial quality values ​​can be obtained in step 220 .

[0115] Optionally, a large language model may be used to determine the initial quality value corresponding to each piece of data. For details, please refer to the second method of the above analysis, which will not be repeated here.

[0116] It is understandable that if the above method 2 is used to calculate the initial quality value, a large language model will be used. However, considering that the accuracy of the calculation result of a large language model is relatively single, therefore, in one solution, multiple large language models can be set. In this case, step 220 can specifically include the following: Figure 3 A flow chart of a method for determining initial quality values ​​corresponding to data under multiple large language models is shown, and the method comprises the following steps:

[0117] Step 310: Using the preset first speech technique, the preset data indicator rules and data are input into the preset multiple large language models.

[0118] The preset first speech is used to indicate the reference quality value of the large language model output. Figure 1 The words shown.

[0119] Step 320: Obtain multiple reference quality values ​​output by multiple preset large language models.

[0120] Step 330: Obtain first weights corresponding to respective preset large language models.

[0121] Optionally, the first weight may be preset or may be adjusted according to actual conditions.

[0122] Step 340: The sum of the products of the first weights and the reference quality values ​​corresponding to the preset multiple large language models is used as the initial quality value of the data.

[0123] For example, suppose there are n models in total, and the first weight of each model is The formula for calculating the initial mass value is as follows:

[0124]

[0125] Among them, e i is the initial quality value of the i-th data, e 1i is the reference quality value of the i-th data under the first model, e 2i is the reference quality value of the i-th data under the second model, and so on.

[0126] Through the above scheme, by integrating the evaluation results of multiple models, the deviation problem caused by single model evaluation can be solved, making the initial quality value more accurate, and allowing the weights of different models to be adjusted according to actual conditions, thereby increasing the flexibility of evaluation.

[0127] Step 230: For each piece of data in the plurality of pieces of data, determine the similarity between the data and other pieces of data in the plurality of pieces of data.

[0128] Here, we first introduce the concept of similarity. Similarity is used to measure the semantic similarity of text data. Generally, the higher the similarity, the more semantically similar the two data are. If the similarity of two data is large, the two data will not be selected repeatedly, such as the following two data:

[0129] Data 1:

[0130] Instructions: Please write a quick sort algorithm in Python

[0131] Output: Python implements the quick sort algorithm as follows: def quick_sort():....

[0132] Data 2:

[0133] Instruction: The quick sort algorithm can quickly sort a disordered array. Please use Python to implement a quick sort code.

[0134] Output: def quick_sort():....The essence of quick sort is to sort by binary search and then merge to get an ordered array.

[0135] From the above data 1 and data 2, it can be seen that data 1 and data 2 express the same meaning, but the way of expression is different, that is, data 1 and data 2 are very similar, and only one of data 1 and data 2 needs to be retained.

[0136] Optionally, there are many ways to calculate the similarity of two data, such as using Euclidean distance to calculate the straight-line distance between two points, calculating the ratio of the intersection and union of two sets to measure their similarity, measuring their similarity by calculating the linear correlation of two variables, etc., which are not limited here.

[0137] Taking the cosine similarity calculation method as an example, before calculating the similarity, the data can be converted into a vector, such as a 1024-dimensional vector, and then the similarity between every two data (that is, every two vectors) is calculated through the cosine similarity of the vector.

[0138] For example, assuming there are 500,000 pieces of data, the i-th piece of data can be converted into the following vector form:

[0139] emb i =[x i1 ,x i2 ,…x i1024 ]

[0140] Among them, emb i A vector representing the i-th data, where the value of i is a positive integer not greater than 500,000.

[0141] Similarly, the vector emb of the j-th data j as follows:

[0142] emb j =[x j1 ,x j2 ,…x j1024 ]

[0143] Then, according to emb i and emb j , calculate the cosine similarity between the i-th data and the j-th data according to the following formula:

[0144] similarity <i,j> =cos <emb i ,emb j >=x i1 *x j1 +x i2 *x j2 +...+x i1024 *x j1024

[0145] Among them, similarity <i,j> Represents the cosine similarity between the i-th data and the j-th data, similarity <i,j> Usually a number between 0 and 1, similarity <i,j> The closer it is to 1, the more similar the i-th data and the j-th data are.

[0146] Optionally, in the above calculation process, the input data can be converted into a 1024-dimensional vector using a model, and the model here can be exemplarily an fm model.

[0147] Optionally, 500,000 initial quality values ​​are obtained in step 220, and then each piece of data is calculated using the formula in step 230 to obtain 500,000*500,000 similarities. To facilitate subsequent calculations, these 500,000*500,000 similarities can be constructed into a similarity matrix.

[0148] For example, the similarity of each piece of data shown in Table 1 below:

[0149] similarity<i,1> similarity<i,2> similarity<i,3> ... similarity<i,50w>

[0150] Table 1

[0151] similarity<i,1> Indicates the similarity between the i-th data and the first data, similarity<i,2> Represents the similarity between the i-th data and the second data, and so on.<i,i> =1, that is, the similarity between the i-th data and the i-th data is 1.

[0152] Furthermore, for 500,000 pieces of data, a 500,000*500,000 similarity matrix can be obtained, for example, the similarity matrix shown in Table 2 below:

[0153]

[0154] Table 2

[0155] Further, optionally, after calculating the similarity of all the data, other data with a large similarity may be deleted according to the similarity between each data and other data.

[0156] For example, there are 500,000 pieces of data in total. In step 240, 500,000*500,000 similarities are obtained. If the size of 500,000*500,000 similarities is compared, the calculation is too complicated. Therefore, for each piece of data, K pieces of data with small similarity can be selected and retained (for example, K=1000), and other pieces of data with large similarity can be deleted. In this way, the subsequent calculation steps can be simplified. For example, the selection can be performed through the following steps:

[0157] {idx_j}=Mintop1000({similarity<i,j> ,j∈[1,2,...50w] and i≠j)

[0158] Among them, Mintop1000 is the similarity of the whole<i,j> Sort from small to large, and finally select the data corresponding to the first 1000 values. idx_j is the jth data among the 1000 data with the smallest similarity to the i-th data. Based on this, the matrix constructed according to the deleted data is less than 500,000*50.

[0159] Step 240: For each piece of data in the plurality of pieces of data, the initial quality value of the data is corrected using the similarity between the data and each other piece of data in the plurality of pieces of data to obtain a final quality value of the data.

[0160] The final quality value is used to indicate the diversity of the instruction data of the data in the instruction data of all the data and the quality of the output result data of the data. Using similarity to correct the initial quality value of the data can be understood as that if the similarity between one data and another data is very large, the initial quality value of the data can be reduced, thereby avoiding the selection of too much similar data. This ensures that the final selected data set maintains both high quality and sufficient diversity.

[0161] From the introduction of similarity in step 230, it can be seen that only one of the two data with high similarity needs to be retained. Therefore, the initial quality value of the data is corrected using the similarity, so that the data with higher quality can be selected and retained from the two data with high similarity.

[0162] Optional, see Figure 4 In the flowchart of determining the final quality value shown in FIG. 2 , step 240 may specifically include the following steps:

[0163] Step 410: Obtain the initial quality value of each other data, and divide the similarity between the data and each other data by the initial quality value of each other data to obtain the correction ratio of each other data to the data.

[0164] Correction ratio k i The calculation formula is as follows:

[0165]

[0166] Among them, e idx_j is the initial quality of the jth data among the K data with the smallest similarity to the i-th data.<i,idx_j> is the similarity between the i-th data and the j-th data.

[0167] Step 420: Calculate the sum of the correction ratios of each other data to the data.

[0168] The sum of the correction ratios k is calculated by the following formula:

[0169] k=∑k i

[0170] Step 430: The product of the sum value and the initial quality value is taken as the final quality value of the data.

[0171] The final quality value is calculated as follows:

[0172] select i =e i *k

[0173] Among them, select i is the final quality value of the i-th data, ei is the initial quality value of the i-th data,

[0174] In this way, the correction ratio is calculated by the similarity between the data and other data, so as to correct the initial quality value, and the data with high quality and small similarity can be screened out.

[0175] By combining steps 410 to 430, the following formula for calculating the final quality value can be obtained:

[0176]

[0177] Step 250: Determine target data among the multiple pieces of data according to the final quality values ​​of the multiple pieces of data.

[0178] Optionally, a threshold can be set to filter out data with a final quality value greater than a first threshold to obtain target data. In this way, the target data obtained is data with a final quality value greater than the first threshold, or a set number of data with the largest final quality value is filtered out as target data.

[0179] Optionally, when multiple data belong to different scenarios (such as analysis, filling, translation, etc. in multiple languages ​​such as python, java, sql, etc.), step 250 can also determine the target data according to the target scenario to which the data belongs. For details, please refer to Figure 5 The method flow chart shown includes the following steps 510 to 540:

[0180] Step 510: Determine the target scenario to which the data belongs.

[0181] For example, there are 500,000 pieces of data in total. Determine the target scenario for each piece of data in the 500,000 pieces of data. Finally, determine that there are n scenarios in the 500,000 pieces of data in total, and the number of data in each scenario is N. j , where j∈[1,2,...n].

[0182] Optionally, the n scenarios may include but are not limited to: python code generation scenario, java code generation scenario, python risk scanning scenario, java risk scanning scenario, etc.

[0183] Step 520: Calculate the sum of the initial quality values ​​of each piece of data included in the target scene, and divide the sum by the total amount of data of the target scene to obtain the average quality value corresponding to the target scene.

[0184] Optionally, the average quality value corresponding to each scene can be calculated according to the following formula:

[0185]

[0186] Among them, Em is the average mass value, e k is the initial quality value of each data in each scenario, ∑e k It is to add up the initial quality values ​​of each piece of data in this scenario.

[0187] Step 530: Divide the initial quality value of the data by the average quality value to obtain a second weight of the data.

[0188] Optionally, the formula for calculating the second weight of the data is as follows:

[0189]

[0190] Among them, e m is the initial quality value of the mth data, w m It is the second weight corresponding to the mth data.

[0191] Step 540: multiply the second weight by the final quality value to obtain the corrected final quality value.

[0192] The final corrected quality value is:

[0193] select m =select1*w m

[0194] Among them, select m is the final quality value after correction, and select1 is the final quality value before correction.

[0195]

[0196] Through the above scheme, data in different scenarios can be mapped to one dimension for quality measurement and comparison. In this way, target data with multiple scenarios can be screened out.

[0197] Through the scheme of steps 210 to 250 above, the initial quality value of the data is determined according to the set data indicator rules, high-quality data can be screened out, and the initial quality value of the data is corrected using similarity, so that data with high similarity can be screened out. In this way, it can be ensured that the target data finally determined is diverse and high-quality data.

[0198] Optionally, after determining the initial quality value corresponding to the data in step 220 and before determining the similarity between the data and other data in the plurality of data in step 230, for details, please refer to Figure 6 The flowchart of using probability to correct the initial quality value shown in FIG. 1 includes the following steps 610 to 640:

[0199] Step 610: Set one or more data indicator rules.

[0200] The data indicator rule may be consistent with the indicator rule in step 220 .

[0201] Step 620: Select a suitable large language model.

[0202] The large language model here can be consistent with the large language model that needs to be fine-tuned, or it can be inconsistent.

[0203] Step 630: Use the preset second speech technique to send the preset data indicator rules and data as target information to the large language model to obtain the probability value output by the large language model;

[0204] The preset second speech is used to indicate the probability value that the data in the target information output by the large language model meets the preset indicator rules in the target information.

[0205] For example, see Figure 7 The following is an example of using a large language model to determine whether the data meets the preset data indicator rules. Figure 7 Taking the above training data 2 as an example, Figure 7 The content is sent to the large language model, and the large language model will output YES or NO and its corresponding probability.

[0206] Step 640: Update the initial quality value using the product of the probability value and the initial quality value.

[0207] Optionally, the formula for correcting the initial quality value using probability is as follows:

[0208] m i =e i *p i

[0209] Among them, e i is the initial mass value, p i is the probability of the large language model output, m i is the corrected initial mass value.

[0210] Through the above scheme, determining the probability that the data meets the preset data indicator rules can make up for the problem of different confidence levels in the data scoring process of the large language model, that is, taking into account the confidence level of the large language model score, making the initial quality value more accurate.

[0211] The above introduces the data processing method. The following is an example of two specific application scenarios to illustrate the above method.

[0212] Application scenario 1: Data processing in a single scenario.

[0213] Step 1: Get 500,000 pieces of data.

[0214] Step 2: Set data indicator rules.

[0215] Step 3: For each of the 500,000 pieces of data, determine the initial quality value corresponding to the data.

[0216] For example, for the first piece of data, using the indicator rule in the above method 2, the initial quality value e1=3.5 corresponding to the first piece of data is obtained.

[0217] Step 4: For each of the 500,000 pieces of data, determine the probability that the data meets the preset data indicator rules.

[0218] Continuing to take the first piece of data as an example, referring to the above steps 410 to 430, it can be obtained that the probability p1=1 that the data meets the preset data indicator rule.

[0219] Step 5: For each of the 500,000 pieces of data, use probability to correct the initial quality value.

[0220] Continuing to take the first piece of data as an example, referring to the above step 440, the corrected initial mass value m1=e1*p1=3.5 is obtained.

[0221] Step 6: For each of the 500,000 pieces of data, determine how similar the data is to other pieces of data.

[0222] Continuing to take the first piece of data as an example, referring to the above step 230, the similarity between the first piece of data and other data can be obtained. Since there are 500,000 pieces of data in total, that is, the first piece of data has a total of 500,000 similarities.

[0223] Step 7: For each of the 500,000 pieces of data, based on the similarity between the data and each other piece of data, obtain the data corresponding to the top 1,000 values ​​with the smallest similarity.

[0224] Sort the similarity of the first data from large to small, and filter the next 1000 data with small similarity to the first data. Specifically, the following formula can be used for filtering:

[0225] {idx_j}=Mintop1000({similarity<i,j> ,j∈[1,2,...50w] and i≠j)

[0226] For example, if we get the 3rd, 5th, 6th, 100th, 200th, etc. data, totaling 1000, then

[0227] {idx_j}={3,5,6,100,200,...}

[0228] Correspondingly, find the initial quality values ​​of the 3rd, 5th, 6th, 100th, 200th, etc., a total of 1000 data, that is, {e3,e5,e6,e 100 ,e 200 ...}.

[0229] Step 8: For each of the 500,000 pieces of data, determine the final quality value of the data based on the initial quality value of the data, the initial quality value of each other data, and the similarity between the data and each other data.

[0230] Assume e3 = 3.1, e5 = 2.8, e6 = 3.0, e 100 =2.9, e 200 =3.6;

[0231] similarity<1,3>=0.99

[0232] similarity<1,5>=0.97

[0233] similarity<1,6>=0.90

[0234] similarity<1,100>=0.89

[0235] similarity<1,200>=0.88

[0236] The final quality value of the data is determined by the following formula:

[0237]

[0238] Here we assume that 5 values ​​are used for calculation demonstration (in fact, there can be more, such as 1000), then

[0239]

[0240] That is, the final quality value of the first piece of data is 0.4136.

[0241] Step 9: Determine the target data based on the final quality value of 500,000 pieces of data.

[0242] Repeat the above steps 3 to 8 for each of the 500,000 data pieces to obtain the final quality value of each data piece, and then sort the final quality values ​​from large to small, and select the data with high final quality values ​​as the target data.

[0243] Optionally, a first threshold may be set to select data with a final quality value greater than the threshold as target data; or a second threshold may be set to filter data in the sorted list between the second threshold values ​​as target data. For example, if 100,000 pieces of data are needed, the first 100,000 pieces of data in the sorted list are selected as target data.

[0244] Application scenario 2: Data processing in multiple scenarios.

[0245] Step 1: Determine the target scenario to which the data belongs.

[0246] For example, among the 500,000 data, there are four scenarios: python instruction generation scenario, java instruction generation scenario, python risk scanning scenario, and java risk scanning scenario. Among them, there are 200,000 data in the python instruction generation scenario, 100,000 data in the java instruction generation scenario, 50,000 data in the python risk scanning scenario, and 150,000 data in the java risk scanning scenario. That is, N python指令生成 =20w, N java指令生成 =10w, N python风险扫描 =5w,

[0247] N java风险扫描 =15w.

[0248] Step 2: Determine the initial quality value of each data contained in each scenario.

[0249] The following introduces the initial quality values ​​of each data in four scenarios: Python instruction generation scenario, Java instruction generation scenario, Python risk scanning scenario, and Java risk scanning scenario.

[0250] 1. Python command generation scene

[0251] Taking four pieces of data in a python instruction generation scenario as an example, the four pieces of data are: python instruction generation 1, python instruction generation 2, python instruction generation 3, and python instruction generation 4.

[0252] The initial qualities of the four pieces of data are calculated according to steps 530 to 550 of the first embodiment.

[0253] For example, the python instruction generates an initial quality value of 1, e1=5, and the probability of meeting the preset data indicator rule is p1=0.9. The initial quality value m1=e1*p1=4.5 is obtained by correcting the initial quality value using the probability.

[0254] The python command generates an initial quality value of 2, e2=4, and the probability of meeting the preset data indicator rules is p2=0.89. The initial quality value m2=e2* is obtained by correcting the initial quality value using probability.

[0255] p2=3.56.

[0256] The python command generates an initial quality value of 3, e3 = 3, and the probability of meeting the preset data indicator rules is p3 = 0.98. The initial quality value obtained by correcting the initial quality value using probability is m3 = e3*

[0257] p3=2.94.

[0258] The python instruction generates an initial quality value of 4, e4=4, and the probability of meeting the preset data indicator rules, p4=0.98. The initial quality value obtained by correcting the initial quality value using probability is m=e4*p4=3.92.

[0259] 2. Java instruction generation scenario

[0260] Taking four pieces of data under the Java instruction generation scenario as an example, the four pieces of data are: Java instruction generation scenario 1, Java instruction generation scenario 2, Java instruction generation scenario 3, and Java instruction generation scenario 4.

[0261] The initial qualities of the four pieces of data are calculated according to steps 530 to 550 of the first embodiment.

[0262] For example, the Java instruction generates an initial quality value of e1=4 for scenario 1, and the probability of meeting the preset data indicator rule is p1=0.95. The initial quality value m1=e1*p1=3.8 is obtained by correcting the initial quality value using the probability.

[0263] The java instruction generates an initial quality value e2=3 for scenario 2, and the probability of meeting the preset data indicator rule p2=0.80. The initial quality value m2=e2*p2=2.4 is obtained by correcting the initial quality value using the probability.

[0264] The java instruction generates an initial quality value e3=3 for scenario 3, and the probability of meeting the preset data indicator rule p3=0.88. The initial quality value m3=e3*p3=2.64 is obtained by correcting the initial quality value using the probability.

[0265] The java instruction generates an initial quality value e4=5 for scenario 4, and the probability of meeting the preset data indicator rule p4=0.98. The initial quality value m4=e4*p4=4.9 is obtained by correcting the initial quality value using the probability.

[0266] 3. Python risk scanning scenario

[0267] Taking the four pieces of data under the python risk scanning scenario as an example, the four pieces of data are: python risk scanning scenario 1, python risk scanning scenario 2, python risk scanning scenario 3, and python risk scanning scenario 4.

[0268] The initial qualities of the four pieces of data are calculated according to steps 530 to 550 of the first embodiment.

[0269] For example, the initial quality value of Python risk scanning scenario 1 is e1=5, and the probability of meeting the preset data indicator rules is p1=0.98. After correcting the initial quality value using probability, the initial quality value obtained is m1=e1*p1=4.9.

[0270] The initial quality value of Python risk scanning scenario 2 is e2=4, and the probability of meeting the preset data indicator rules is p2=0.82. The initial quality value obtained by correcting the initial quality value using probability is m2=e2*p2=3.28.

[0271] The initial quality value of Python risk scanning scenario 3 is e3=3, and the probability of meeting the preset data indicator rules is p3=0.78. The initial quality value m3=e3*p3=2.34 is obtained by correcting the initial quality value using probability.

[0272] The initial quality value of Python risk scanning scenario 4 is e4=4, and the probability of meeting the preset data indicator rules is p4=0.98. The initial quality value m4=e4*p4=3.92 is obtained by correcting the initial quality value using probability.

[0273] 4. Java risk scanning scenario

[0274] Taking four pieces of data under the Java risk scanning scenario as an example, the four pieces of data are: Java risk scanning scenario 1, Java risk scanning scenario 2, Java risk scanning scenario 3, and Java risk scanning scenario 4.

[0275] The initial qualities of the four pieces of data are calculated according to steps 530 to 550 of the first embodiment.

[0276] For example, the initial quality value of Java risk scanning scenario 1 is e1=3, and the probability of meeting the preset data indicator rules is p1=0.98. The initial quality value m1=e1*p1=2.94 is obtained by correcting the initial quality value using probability.

[0277] The initial quality value of Java risk scanning scenario 2 is e2=4, and the probability of meeting the preset data indicator rules is p2=0.82. The initial quality value m2=e2*p2=3.28 is obtained by correcting the initial quality value using probability.

[0278] The initial quality value of Java risk scanning scenario 3 is e3=5, and the probability of meeting the preset data indicator rules is p3=0.98. The initial quality value m3=e3*p3=4.9 is obtained by correcting the initial quality value using probability.

[0279] The initial quality value of Java risk scanning scenario 4 is e4=4, and the probability of meeting the preset data indicator rules is p4=0.88. The initial quality value m4=e4*p4=3.52 is obtained by correcting the initial quality value using probability.

[0280] Step 3: Determine the average quality value corresponding to each scene based on the total amount of data in the target scene and the initial quality value of each data contained in each scene.

[0281] The average quality of the scenes generated by the python command is:

[0282]

[0283] The average quality value in the Java instruction generation scenario is:

[0284]

[0285] The average quality scores of the Python risk scan scenarios are:

[0286]

[0287] The average quality values ​​of the Java risk scanning scenarios are:

[0288]

[0289] Step 4: For each of the 500,000 pieces of data, calculate the second weight of the data based on the average quality value and the initial quality value of the data.

[0290] For example, the initial quality value of Python instruction generation 1 calculated in step 620 is 4.5, and the average quality value of Python instruction generation scene calculated in step 630 is 3.72, then the second weight of Python instruction generation 1 is:

[0291]

[0292] Step 5: For each of the 500,000 pieces of data, determine how similar the data is to other pieces of data.

[0293] Continuing to take Python instruction generation 1 as an example, refer to step 560 in the above embodiment 1 to determine the similarity between Python instruction generation 1 and other data.

[0294] Step 6: For each of the 500,000 pieces of data, based on the similarity between the data and other pieces of data, delete the other pieces of data with high similarity.

[0295] Continuing to take Python instruction generation 1 as an example, referring to step 570 in the above embodiment 1, determine the data with a small similarity to Python instruction generation 1, for example, there are 1000 data.

[0296] Taking the five data with small similarity of python instruction generation 1 as an example, the five data are: python instruction generation 4, python instruction generation 2, Java risk scan 1, python risk scan 2, Java instruction generation 2.

[0297] Assume that the similarity values ​​of these five data and the python instruction to generate 1 are:

[0298] The similarity between python instruction generation 4 and python instruction generation 1 is 0.9;

[0299] The similarity between python instruction generation 2 and python instruction generation 1 is 0.88;

[0300] The similarity between Java risk scanning 1 and python instruction generation 1 is 0.85;

[0301] The similarity between python risk scanning 2 and python instruction generation 1 is 0.82;

[0302] The similarity between Java instruction generation 2 and Python instruction generation 1 is 0.75.

[0303] Step 7: For each of the 500,000 pieces of data, determine the final quality value of the data based on the initial quality value of the data, the initial quality value of each other data, and the similarity between the data and each other data.

[0304] Continuing to take Python instruction generation 1 as an example, referring to step 580 in the above embodiment 1, the final quality value of Python instruction generation 1 is:

[0305]

[0306] Step 8: For each of the 500,000 pieces of data, use the second weight to correct the final quality value.

[0307] Continuing to take Python instruction generation 1 as an example, in the above step 640, the second weight of Python instruction generation 1 is calculated to be 1.2, and the final quality value of Python instruction generation 1 is:

[0308]

[0309]

[0310] Step 9: Determine the target data based on the final quality value of 500,000 pieces of data.

[0311] Based on the same concept, an embodiment of the present application also provides a data processing device, which can execute the testing method introduced in the above content.

[0312] See also Figure 8 , a schematic diagram of the structure of a data processing device provided in an embodiment of the present application is given, such as Figure 8 As shown, the data processing device includes:

[0313] The acquisition module 801 is used to acquire multiple pieces of data, each of which includes instruction data and output result data, and the output result data is obtained by analyzing the instruction data using a large language model.

[0314] An initial quality value determination module 802 is used to determine, for each of the plurality of data, an initial quality value of the data according to a preset data indicator rule, wherein the initial quality value of the data is used to indicate a quality value of output result data in the data, and the higher the degree of correspondence between the output result data in the data and the instruction data in the data, the greater the quality value of the output result data in the data;

[0315] A similarity determination module 803 is used to determine, for each piece of data in the plurality of pieces of data, a similarity between the data and each other piece of data in the plurality of pieces of data;

[0316] The correction module 804 is used to correct the initial quality value of each piece of data among the multiple data using the similarity between the data and each other piece of data among the multiple data to obtain a final quality value of the data, wherein the final quality value is used to indicate the diversity of the instruction data of the data among the instruction data of all data and the quality of the output result data of the data.

[0317] The target data determination module 805 is used to determine the target data among the multiple data according to the final quality values ​​of the multiple data; the target data is the data whose final quality value is greater than the first threshold, or the target data is a set number of data with the largest final quality value.

[0318] In one possible implementation, the initial quality value determination module 802 is specifically used to: use a preset first wording, input preset data indicator rules and data into multiple preset large language models, obtain multiple reference quality values ​​output by the multiple preset large language models, the preset first wording is used to instruct the large language model to output the reference quality value, obtain the first weight corresponding to each of the multiple preset large language models, and use the sum of the products of the first weight corresponding to each of the multiple preset large language models and the reference quality value as the initial quality value of the data.

[0319] In a possible implementation, it also includes an initial quality value updating module 806, which is used to use a preset second wording to send preset data indicator rules and data as target information to the large language model to obtain the probability value output by the large language model. The preset second wording is used to indicate the probability value that the data in the target information output by the large language model meets the preset indicator rules in the target information, and the product of the probability value and the initial quality value is used to update the initial quality value.

[0320] In one possible implementation, the correction module 804 is specifically used to: obtain the initial quality value of each other data, and divide the similarity between the data and each other data by the initial quality value of each other data to obtain the correction ratio of each other data to the data, calculate the sum of the correction ratios of each other data to the data, and use the product of the sum and the initial quality value as the final quality value of the data.

[0321] In one possible implementation, the preset data indicator rules include one or more of the following: the code comments meet the specification requirements, the actual completed answers are consistent with the requirements of the instruction data, the generated answers are correct and can be successfully executed, the generated answers are fluent and unambiguous, the format of the generated answers meets the requirements, and the format of the generated answers can fully explain the answers.

[0322] In one possible implementation, multiple data belong to N scenes, where N is an integer greater than or equal to 2. After obtaining the final quality value of the data, the target data determination module 805 is also used to: determine the target scene to which the data belongs, calculate the sum of the initial quality values ​​of the data contained in the target scene, divide the sum by the total amount of data in the target scene to obtain the average quality value corresponding to the target scene, divide the initial quality value of the data by the average quality value to obtain a second weight of the data, and use the product of the second weight and the final quality value as the corrected final quality value.

[0323] In a possible implementation, the N scenarios include one or more of the following: a python code generation scenario, a java code generation scenario, a python risk scanning scenario, and a java risk scanning scenario.

[0324] In a possible implementation, a deletion module 807 is further included, which is used to delete other data with a large similarity according to the similarity between the data and other data.

[0325] See also Fig. 9 , showing a schematic diagram of the structure of another data device provided in an embodiment of the present application, such as Fig. 9 As shown, the test device includes: a memory 901 and a processor 902, and the processor 902 is coupled to the memory 901. The memory 901 is used to store program instructions, and the processor 902 is used to call the program instructions stored in the memory 901 and execute the above data processing method according to the obtained program.

[0326] Optionally, the data processing device may further include an interface circuit 903, which may be a transceiver or an input / output interface. The input / output interface is used to input and / or output information, where output may be understood as sending and input may be understood as receiving. The processor 902 may communicate with other devices in the test device or other devices other than the test device through the interface circuit 903 to obtain information required for executing the above data processing method.

[0327] When the data processing device 900 is used to implement Figure 2 In the method shown, the processor 902 is used to implement the functions of the above-mentioned acquisition module 801, initial quality value determination module 802, similarity determination module 803, correction module 804, and target data determination module 805.

[0328] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0329] The memory in the embodiments of the present application can be a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a register, a hard disk, a mobile hard disk, a compact disc read-only memory (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. The storage medium can also be a component of the processor.

[0330] Based on the same technical concept, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the computer executes the above-mentioned transaction processing method.

[0331] Based on the same technical concept, an embodiment of the present invention further provides a computer-readable program product, which, when executed, enables a computer to execute the above transaction processing method.

[0332] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0333] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0334] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0335] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0336] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire multiple pieces of data, each piece of the multiple pieces of data includes instruction data and output result data, and the output result data is obtained by analyzing the instruction data using a large language model; For each of the plurality of data, determining an initial quality value of the data according to a preset data indicator rule, wherein the initial quality value of the data is used to indicate a quality value of output result data in the data, and the higher the degree of correspondence between the output result data in the data and the instruction data in the data, the greater the quality value of the output result data in the data; For each piece of data in the plurality of pieces of data, determining a similarity between the data and each other piece of data in the plurality of pieces of data; For each piece of data in the plurality of pieces of data, using the similarity between the data and each other piece of data in the plurality of pieces of data, an initial quality value of the data is corrected to obtain a final quality value of the data, wherein the final quality value is used to indicate the diversity of the instruction data of the data in the instruction data of all data and the quality of the output result data of the data; According to the final quality values ​​of the multiple pieces of data, target data among the multiple pieces of data are determined; the target data is data whose final quality value is greater than a first threshold, or the target data is a set number of data with the largest final quality value.

2. The method according to claim 1, characterized in that For each piece of the plurality of pieces of data, determining an initial quality value corresponding to the data according to a preset data indicator rule includes: Using a preset first speech technique, inputting the preset data indicator rule and the data into a plurality of preset large language models to obtain a plurality of reference quality values ​​output by the plurality of preset large language models, wherein the preset first speech technique is used to instruct the large language model to output the reference quality value; Obtaining first weights corresponding to each of the plurality of preset large language models; The sum of the products of the first weights and the reference quality values ​​corresponding to the preset multiple large language models is used as the initial quality value of the data.

3. The method according to claim 1, characterized in that After determining the initial quality value of the data according to the preset data indicator rule and before determining the similarity between the data and each other data in the plurality of data, the method further includes: Using a preset second speech, sending the preset data indicator rule and the data as target information to the large language model, obtaining a probability value output by the large language model, wherein the preset second speech is used to instruct the large language model to output a probability value that the data in the target information meets the preset indicator rule in the target information; The initial quality value is updated using the product of the probability value and the initial quality value.

4. The method according to any one of claims 1 to 3, characterized in that The method of correcting the initial quality value of each piece of data in the plurality of pieces of data by using the similarity between the data and each other piece of data in the plurality of pieces of data to obtain the final quality value of the data includes: Obtaining an initial quality value of each other data, and dividing the similarity between the data and each other data by the initial quality value of each other data, to obtain a correction ratio of each other data to the data; The sum of the correction ratios of the other data to the data is calculated, and the product of the sum and the initial quality value is used as the final quality value of the data.

5. The method according to any one of claims 1 to 3, characterized in that The preset data indicator rules include one or more of the following: The code comments meet the specification requirements, the actual completed answers are consistent with the requirements of the instruction data, the generated answers are correct and can be executed successfully, the generated answers are fluent and unambiguous, the format of the generated answers meets the requirements, and the format of the generated answers can fully explain the answers.

6. The method according to any one of claims 1 to 3, characterized in that The multiple pieces of data belong to N scenes, where N is an integer greater than or equal to 2; After obtaining the final quality value of the data, the method further includes: Determining a target scenario to which the data belongs; Calculate the sum of the initial quality values ​​of the various pieces of data included in the target scene, and divide the sum by the total amount of data of the target scene to obtain an average quality value corresponding to the target scene; Dividing the initial quality value of the data by the average quality value to obtain a second weight of the data; The product of the second weight and the final quality value is used as the corrected final quality value.

7. The method according to claim 6, characterized in that The N scenarios include one or more of the following: Python code generation scenario, Java code generation scenario, Python risk scanning scenario, Java risk scanning scenario.

8. A data processing device, characterized in that: The data processing device comprises: An acquisition module, used to acquire multiple pieces of data, each of which includes instruction data and output result data, and the output result data is obtained by analyzing the instruction data using a large language model; an initial quality value determination module, configured to determine, for each of the plurality of data, an initial quality value of the data according to a preset data indicator rule, wherein the initial quality value of the data is used to indicate a quality value of output result data in the data, and the higher the degree of correspondence between the output result data in the data and the instruction data in the data, the greater the quality value of the output result data in the data; A similarity determination module, configured to determine, for each piece of data in the plurality of pieces of data, a similarity between the data and each other piece of data in the plurality of pieces of data; a correction module, configured to correct, for each piece of data in the plurality of data, an initial quality value of the data by using a similarity between the data and each other piece of data in the plurality of data, to obtain a final quality value of the data, wherein the final quality value is used to indicate the diversity of the instruction data of the data in the instruction data of all data and the quality of the output result data of the data; The target data determination module is used to determine the target data among the multiple data according to the final quality values ​​of the multiple data; the target data is the data whose final quality value is greater than a first threshold, or the target data is a set number of data with the largest final quality value.

9. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device, and when the program is run on the computer device, the computer device executes the steps of any method described in claims 1 to 7.

10. A computer program product, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images