Conflict repair method and device for relational training data, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a conflict repair method and device for relational training data, equipment and a medium. According to the method, key attribute identification is carried out on the to-be-repaired training number, performance gains of different attribute combinations after repair are evaluated under the condition that the model is not retrained, which specific attributes should be preferentially repaired are determined, key sample identification is carried out on the to-be-repaired training number, and on the basis of the influence on model parameters before and after sample repair, the performance gains of the different attribute combinations after repair are evaluated. It is determined which specific training samples should be preferentially repaired. According to the method and the device, the repair process can be concentrated on the most influential fields and samples instead of performing comprehensive repair on all attributes and all samples, so that a large amount of low-income modification operation is avoided, the repair process has clear pertinence and selectivity, and the repair cost of the training data is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and medium for conflict repair of relational training data. Background Technology

[0002] During the training of machine learning models, model performance is highly dependent on the accuracy and consistency of the training data. In practical applications, relational training data typically originates from multiple business systems or data acquisition channels, such as customer management systems, transaction systems, and manually entered databases. Due to frequent data updates, complex structures, and heterogeneous sources, conflicts are easily generated during integration and maintenance, including inconsistent attribute values, violation of constraints, and semantic contradictions across tables. These conflicts can interfere with parameter learning during the model training phase, causing the model's feature distribution to deviate from the true pattern, thereby leading to decreased predictive performance or insufficient generalization ability.

[0003] Existing conflict resolution (CR) methods are mostly based on manual rules, integrity constraints, or statistical detection mechanisms, typically employing full-scale remediation or partial cleaning based on anomaly scores. Performing full-scale remediation on large-scale multi-table datasets requires traversing, comparing, and modifying all records, which is extremely costly. Therefore, reducing remediation costs is a pressing issue that needs to be addressed during conflict data remediation. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, device, and medium for conflict repair of relational training data, in order to solve the problem of high repair costs in the process of conflict data repair.

[0005] In a first aspect, embodiments of this application provide a conflict resolution method for relational training data, the conflict resolution method comprising: Obtain training data to be repaired and a target model. The training data to be repaired is used to train the target model. The training data to be repaired includes multiple attributes and multiple samples. Obtain the error information of the target model on the training data to be repaired, and filter the training data to be repaired based on the error information to obtain an evaluation set; Based on the training data to be repaired and the evaluation set, the remaining training data to be repaired is determined, and conflict repair is performed on the remaining training data to be repaired to obtain pre-repaired data. Multiple candidate attribute subsets are obtained. For any candidate attribute subset, the attribute-level performance gain when the candidate attribute subset is repaired is calculated based on the target model, the evaluation set and the pre-repair data. All candidate attribute subsets are traversed to obtain the attribute-level performance gain of each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, key attributes are identified and processed to obtain attribute-level repair data. For any sample, the sample-level performance gain during sample repair is calculated based on the target model, the evaluation set, the remaining training data to be repaired, and the pre-repair data. Based on the sample-level performance gain, key samples are identified and processed to obtain sample-level repair data. The attribute-level repair data and the sample-level repair data are conflict-corrected to obtain the repaired data.

[0006] Secondly, embodiments of this application provide a conflict resolution device for relational training data, the conflict resolution device comprising: The acquisition module is used to acquire the training data to be repaired and the target model. The training data to be repaired is used to train the target model. The training data to be repaired includes multiple attributes and multiple samples. A filtering module is used to obtain error information of the target model on the training data to be repaired, and to filter the training data to be repaired based on the error information to obtain an evaluation set. The pre-repair module is used to determine the remaining training data to be repaired based on the training data to be repaired and the evaluation set, and to perform conflict repair on the remaining training data to be repaired to obtain pre-repaired data. The first identification module is used to acquire multiple candidate attribute subsets. For any candidate attribute subset, the module calculates the attribute-level performance gain when the candidate attribute subset is repaired based on the target model, the evaluation set, and the pre-repair data. The module iterates through all candidate attribute subsets to obtain the attribute-level performance gain of each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, the module identifies and processes key attributes to obtain attribute-level repair data. The second identification module is used to calculate the sample-level performance gain when repairing the sample for any sample based on the target model, the evaluation set, the remaining training data to be repaired, and the pre-repair data, and to identify key samples based on the sample-level performance gain to obtain sample-level repair data. The repair module is used to resolve conflicts between the attribute-level repair data and the sample-level repair data to obtain the repaired data.

[0007] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the conflict resolution method as described above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the conflict resolution method described above.

[0009] The advantages of this application compared to the prior art are: This application identifies key attributes in the training data to be repaired, and evaluates the performance gain of different attribute combinations after repair without retraining the model. It also determines which specific attributes should be prioritized for repair. Furthermore, by identifying key samples in the training data to be repaired, it determines which specific training samples should be prioritized for repair based on the impact of sample repair on model parameters. This allows the repair process to focus on the most influential fields and samples, rather than performing comprehensive repair on all attributes and all samples, avoiding numerous low-return modifications. This makes the repair process more targeted and selective, reducing the cost of repairing training data. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating the application environment of a conflict resolution method for relational training data provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a conflict resolution method for relational training data according to an embodiment of this application. Figure 3 This is a schematic diagram of the structure of a conflict repair device for relational training data provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0015] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0016] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0017] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0018] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0019] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0020] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0021] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0022] To illustrate the technical solution of this application, specific embodiments are described below.

[0023] One embodiment of this application provides a conflict resolution method for relational training data, which can be applied to, for example... Figure 1In this application environment, the client communicates with the server. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0024] To illustrate the technical solution of this application, specific embodiments are described below.

[0025] See Figure 2 This is a flowchart illustrating a conflict resolution method for relational training data according to an embodiment of this application. Figure 2 As shown, the conflict resolution method for relational training data may include the following steps.

[0026] S201: Obtain the training data to be repaired and the target model. The training data to be repaired is used to train the target model. The training data to be repaired includes multiple attributes and multiple samples.

[0027] In step S201, the training data to be repaired is labeled data, a relational dataset used to train the target model. It contains multiple attributes describing entity features and multiple specific entity samples. The target model is a machine learning model or deep learning model built based on specific business scenarios or task requirements, such as a classification model, regression model, recommendation model, etc. The training data to be repaired is used to train the target model.

[0028] In this embodiment, the training data to be repaired and the target model are acquired. The training data to be repaired is data from relational datasets, such as the Adult, Nursery, Bank, Default, German, and RoadSafety datasets. These datasets all contain structured tabular data, and each dataset corresponds to a different business scenario. For example, the Adult dataset mainly involves demographic and income-related attributes and samples, the Nursery dataset revolves around assessment information for nurseries, and the Bank dataset contains marketing-related data for bank customers. The target model is determined according to actual application needs. For example, when training with the Adult dataset, the target model may be a classification model used to predict whether an individual's income exceeds a specific threshold; when using the Bank dataset, the target model may be a classification model to determine whether a customer will respond to bank marketing activities. By acquiring these specific training data to be repaired and the corresponding target models, clear processing objects and application scenarios are provided for subsequent data conflict repair.

[0029] The training data to be repaired includes multiple attributes and multiple samples. Each sample consists of several attribute values, which together describe the feature information of the sample in different dimensions. For example, in the Adult dataset, attributes may include age, job type, education level, marital status, occupation, race, gender, and capital gains, while each sample corresponds to a specific individual, and the values ​​of its various attributes together constitute the complete data record of that individual.

[0030] S202: Obtain the error information of the target model on the training data to be repaired, and filter the training data to be repaired based on the error information to obtain the evaluation set.

[0031] In step S202, the error information includes sample loss value, prediction confidence and output stability. Based on the error information, the training data to be repaired is screened, and the data whose error information meets the preset requirements are selected to obtain the evaluation set.

[0032] In this embodiment, error information of the target model on the training data to be repaired is obtained. This error information includes sample loss, prediction confidence, and output stability. The sample loss is the supervised loss of the training data to be repaired under the current target model. This is calculated by inputting the training data to be repaired into the target model, outputting a predicted value, and then calculating the corresponding supervised loss based on the predicted value and the label value of the training data to be repaired. Prediction confidence is a measure of the credibility of the prediction result output by the target model after predicting the input training data to be repaired. Output stability is the variance of the results obtained from multiple forward predictions after applying a small, fixed-amplitude perturbation to the training data to be repaired.

[0033] Based on the error information, the training data to be repaired is filtered to obtain an evaluation set. During the filtering process, the data can be filtered sequentially based on sample loss value, prediction confidence, and output stability. For example, first, training data to be repaired that meets a preset requirement for sample loss value is filtered out. Then, training data to be repaired that meets a preset requirement for prediction confidence is filtered out from the training data to be repaired that meets the preset requirement for sample loss value. Finally, training data to be repaired that meets a preset requirement for prediction confidence is filtered out from the training data to be repaired that meets the preset requirement for output stability. Other methods can also be used for filtering; this embodiment does not limit the scope of the method.

[0034] Optionally, the training data to be repaired is filtered based on error information to obtain an evaluation set, including: Obtain the predicted category of the training data to be repaired, and classify the training data to be repaired according to the predicted category to obtain the category of training data to be repaired for each category. For any category, obtain the attribute values ​​of the category-specific training data to be repaired, and perform stratification processing on the category-specific training data to be repaired according to the attribute values ​​to obtain the stratified training data to be repaired for each layer. For any given layer, the training data to be repaired is filtered based on the error information to obtain the set of training data to be repaired after filtering for each layer. This process is repeated for all layers to obtain the set of training data to be repaired after filtering for all layers. The set of training data to be repaired after filtering for all layers is then determined as the evaluation set.

[0035] In this embodiment, the predicted category of the training data to be repaired is obtained. Based on the predicted category, the training data to be repaired is divided into categories, resulting in training data to be repaired corresponding to each category. The predicted category is the category prediction value output by each piece of training data to be repaired after it is input into the target model. For example, if the predicted category value includes 0, 1, 2, etc., the training data to be repaired is divided into categories, resulting in training data to be repaired corresponding to three categories.

[0036] For any given category, obtain the attribute values ​​of the training data to be repaired for that category. Based on these attribute values, stratify the training data according to the category. If the attribute value is discrete, such as a / b / c, then stratify by a / b / c; if the attribute value is continuous, then stratify by interval. The final stratification is a combination of the predicted category value and the attribute value. For example: (predicted category value is 0, attribute A value is a1), (predicted category value is 1, attribute A value is a2), each combination corresponds to a stratification.

[0037] For any given layer, the training data to be repaired in each layer is filtered based on error information. Specifically, the training data to be repaired in each layer is filtered based on sample loss value, prediction confidence, and output stability. For example, in each layer, training data to be repaired that meets the preset requirements for sample loss value is first selected. Then, training data to be repaired that meets the preset requirements for prediction confidence is selected from the training data to be repaired that meets the preset requirements for sample loss value. Finally, training data to be repaired that meets the preset requirements for output stability is selected from the training data to be repaired that meets the preset requirements for prediction confidence.

[0038] In this embodiment, the training data to be repaired is classified into categories, and within each category, the training data to be repaired is further stratified according to the attribute values. This ensures that the evaluation set includes data from each category and that representative training data is taken from each stratum, without biasing the entire evaluation set towards a certain category or attribute value, thereby forming a relatively correct and stable evaluation set.

[0039] Optionally, the hierarchical training data to be repaired is filtered based on error information to obtain a set of training data to be repaired after filtering at each level, including: Based on the prediction confidence and output stability, the training data to be repaired is initially screened to obtain the initial screening results. The initial screening results are sorted from largest to smallest based on their sample loss values, and the initial screening results that fall within a preset range are determined as the evaluation set.

[0040] In this embodiment, the training data to be repaired is initially screened based on prediction confidence and output stability. Specifically, training data that does not meet the requirements is filtered out based on output stability and prediction confidence, resulting in an initial screening result. For example, training data with variance greater than a preset variance threshold is filtered out based on output stability. Then, training data with prediction probability values ​​less than a preset probability threshold is filtered out from the remaining training data to be repaired, resulting in an initial screening result. The initial screening result is then sorted from largest to smallest based on sample loss values, and the initial screening result within a preset range is determined as the evaluation set. For example, the initial screening result where the sample loss values ​​of the training data to be repaired are in the 30%-70% range is determined as the evaluation set.

[0041] S203: Based on the training data to be repaired and the evaluation set, determine the remaining training data to be repaired, perform conflict repair on the remaining training data to be repaired, and obtain pre-repaired data.

[0042] In step S203, the remaining training data to be repaired is the training data remaining after subtracting the training data to be repaired from the training data to be repaired in the evaluation set. The pre-repaired data is the data after conflict repair of the remaining training data to be repaired based on the conflict repair algorithm.

[0043] In this embodiment, the training data to be repaired is subtracted from the training data to be repaired in the evaluation set to obtain the remaining training data to be repaired. Conflict repair is then performed on the remaining training data to obtain pre-repaired data. The conflict repair is performed on the remaining training data based on a conflict repair algorithm, such as a conflict repair algorithm based on rule-based reasoning or machine learning model prediction, etc., which is not limited in this embodiment. If rule-based reasoning is used, a series of conflict resolution rules for different data relationship types can be predefined, such as "when two attribute values ​​of the same entity are contradictory, the data record with higher source credibility shall prevail" and "for causal relationship conflicts, attribute value combinations that conform to domain common sense or business logic shall be retained first," etc. Then, the conflicting tuples in the remaining training data to be repaired are automatically repaired according to these rules, and the final repair value of each attribute in each conflicting tuple is determined, thereby generating pre-repaired data. If machine learning model prediction is used, a conflict repair prediction model can be trained using labeled conflict-free samples or historical repair cases. The conflicting tuples and their contextual features in the remaining training data to be repaired are input into the model, and the model outputs the optimal repair value suggestions for each conflicting attribute, thereby obtaining pre-repaired data.

[0044] S204: Obtain multiple candidate attribute subsets. For any candidate attribute subset, calculate the attribute-level performance gain when repairing the candidate attribute subset based on the target model, evaluation set, and pre-repair data. Traverse all candidate attribute subsets to obtain the attribute-level performance gain of each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, identify and process key attributes to obtain attribute-level repair data.

[0045] In step S204, the candidate attribute subset is a non-empty subset obtained after multiple rounds of crossover and mutation processing. This subset can contain a single attribute or multiple attributes. Attribute-level performance gain characterization, based on the target model's training and prediction using pre-repaired data, involves further conflict repair on conflicting tuples within a specific attribute subset of the evaluation set. The prediction performance of the target model on this evaluation set is then assessed based on the attribute-level performance gain. Key attribute identification processing is then performed on the evaluation set to obtain attribute-level repaired data.

[0046] In this embodiment, any subset of attributes is obtained. For example, if the training data contains three attributes: "name", "age", and "position", the subset of attributes can be {"age"}, {"name", "position"}, or any combination thereof, but does not include an empty set. Each subset of attributes represents a set of attributes that are considered for joint or individual repair during the conflict resolution process. Its composition must cover all possible combinations of attributes that may affect data quality and subsequent model training.

[0047] Based on the target model, evaluation set, and pre-repair data, the attribute-level performance gain when repairing candidate attribute subsets is calculated. When calculating the attribute-level performance gain when repairing candidate attribute subsets, it can be determined by training a lightweight model and based on the performance gain of the lightweight model on the candidate attribute subsets.

[0048] Traverse all candidate attribute subsets to obtain the attribute-level performance gain for each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, identify and process the key attributes to obtain attribute-level repair data. For example, extract the candidate attribute subsets corresponding to the top a attribute-level performance gains with the largest attribute-level performance gains as key attribute subsets, and determine the training data corresponding to the key attribute subsets in the evaluation set as attribute-level repair data.

[0049] Optionally, multiple subsets of candidate attributes can be obtained, including: Obtain multiple initial candidate attribute subsets for the current round. For any initial candidate attribute subset, calculate the initial attribute-level performance gain when repairing the initial candidate attribute subset based on the target model, evaluation set, and pre-repair data. Traverse all initial candidate attribute subsets to obtain the attribute-level performance gain for each initial candidate attribute subset. Select the first m corresponding initial candidate attribute subsets from the attribute-level performance gain values ​​as the seed candidate attribute subset; The seed candidate attribute subset is subjected to crossover and mutation processing to obtain a new candidate attribute subset. The new candidate attribute subset is used as multiple initial candidate attribute subsets for the next round until the preset round is reached, and the final new candidate attribute subset is obtained. The final new candidate attribute subset is determined as multiple candidate attribute subsets.

[0050] In this embodiment, multiple initial candidate attribute subsets for the current round are obtained. For any initial candidate attribute subset, the initial attribute-level performance gain when the initial candidate attribute subset is repaired is calculated based on the target model, evaluation set and pre-repair data. All initial candidate attribute subsets are traversed to obtain the attribute-level performance gain of each initial candidate attribute subset. The first m initial candidate attribute subsets corresponding to the attribute-level performance gain values ​​are determined as seed candidate attribute subsets, where m is a positive integer. Crossover and mutation processing is performed on the seed attribute subsets to obtain new candidate attribute subsets. Crossover and mutation processing involves randomly selecting two different attribute subsets from the seed candidate attribute subsets as parent attribute subsets, and exchanging and combining the attributes of these two parent attribute subsets. For example, the first half of the attributes of the first parent attribute subset is concatenated with the second half of the attributes of the second parent attribute subset to form a new attribute subset. This is the crossover processing. Mutation processing involves randomly changing one or more attributes of the new attribute subset obtained after crossover processing or a certain attribute subset in the original candidate attribute subset with a preset mutation probability. For example, replacing an attribute originally included in the attribute subset with another attribute not included in the attribute set, removing an attribute from the attribute subset, or adding a new attribute to the attribute subset. Through such crossover and mutation operations, the seed candidate attribute subset is expanded and optimized, thereby generating new candidate attribute subsets.

[0051] The new candidate attribute subset is used as multiple initial candidate attribute subsets for the next round. The process of "for any initial candidate attribute subset, calculating the initial attribute-level performance gain for repair based on the target model, evaluation set, and pre-repair data; traversing all initial candidate attribute subsets to obtain the attribute-level performance gain for each subset; selecting the top m initial candidate attribute subsets with the attribute-level performance gain values ​​as seed candidate attribute subsets, where m is a positive integer; performing crossover and mutation on the seed candidate attribute subsets to obtain new candidate attribute subsets" is repeated. This process is repeated for each round until a preset number of rounds is reached. The final new candidate attribute subset is then defined as multiple candidate attribute subsets, i.e., the new candidate attribute subsets obtained in the preset target rounds are defined as multiple candidate attribute subsets. For example, the new candidate attribute subset obtained in the 5th round is defined as multiple candidate attribute subsets.

[0052] In another embodiment, it can also be determined whether it is the final round based on the attribute-level performance gain value of the new candidate attribute subset. For example, if the mean of the attribute-level performance gain of the new candidate attribute subset is less than a preset threshold, then the new candidate attribute subset of the corresponding round is determined as multiple candidate attribute subsets.

[0053] In this embodiment, candidate attribute subsets are subjected to crossover and mutation processing to obtain new candidate attribute subsets. Crossover operation enables information exchange and fusion between different candidate attribute subsets, while mutation operation introduces new attribute combination possibilities while retaining the main features of the candidate attribute subsets, effectively avoiding the algorithm from getting stuck in local optima. Through this synergistic effect of crossover and mutation, the diversity of candidate attribute subsets can be rapidly expanded, the probability of finding the globally optimal attribute subset can be increased, and ultimately the accuracy and efficiency of relational training data conflict repair can be improved.

[0054] Optionally, based on the target model, evaluation set, and pre-repair data, the initial attribute-level performance gain for repairing the initial candidate attribute subset is calculated, including: Determine the set of pre-repair attribute data in the pre-repair data from the initial subset of candidate attributes; Obtain a preset lightweight model, train the lightweight model using a pre-repaired attribute dataset, and obtain a trained lightweight model. Based on the pre-repair attribute dataset, calculate the residual of the target model on the pre-repair attribute dataset, and fit the residual using the trained lightweight model to obtain the residual fitting model. The residual fitting model is added to the target model to construct the evaluation model; Based on the evaluation set, the first performance gain of the target model on the evaluation set and the second performance gain of the evaluation model on the evaluation set are calculated. The difference between the second performance gain and the first performance gain is calculated to obtain the attribute-level performance gain.

[0055] In this embodiment, the initial candidate attribute subset is determined as a pre-repair attribute data set within the pre-repair data; that is, the pre-repair data contains training data for the initial candidate attribute subset. A preset lightweight model is obtained, and the lightweight model is trained using the pre-repair attribute data set to obtain a trained lightweight model. The preset lightweight model can be a lightweight tree model such as a gradient boosting tree. When training the lightweight model using the pre-repair attribute data set, attribute features can be extracted from the pre-repair attribute data set. These attribute features are data representations extracted from the training data of the pre-repair attribute data set that reflect the essential characteristics of the initial candidate attribute subset. Examples include numerical distribution features of the attributes (such as mean, variance, extreme values, etc.), category distribution features (such as the proportion and frequency of each category), and correlation features between attributes (such as correlation coefficient, co-occurrence probability, etc.). The lightweight model is then trained based on these attribute features to obtain a trained lightweight model.

[0056] Based on the pre-repaired attribute dataset, the residual of the target model on the pre-repaired attribute dataset is calculated, which is the difference between the true label and the prediction result of the target model. The residual is then fitted using a pre-trained lightweight model to obtain a residual-fitted model. Specifically, the residual calculation involves iterating through each training data point in the pre-repaired attribute dataset, inputting that training data point into the target model to obtain the corresponding prediction result, and then subtracting the prediction result from the true label of that training data point to obtain the residual for that training data point. This process is repeated for each training data point to obtain a residual set. This residual set is then used as the training objective, and the attribute features from the pre-repaired attribute dataset are used as input to perform a secondary training on the pre-trained lightweight model. This allows the lightweight model to learn the mapping relationship from attribute features to residuals, meaning the output of the trained lightweight model should be as close as possible to the residual corresponding to the training data point. Ultimately, this results in a residual-fitted model that accurately captures the prediction bias patterns of the target model.

[0057] The residual fitting model is added to the target model to construct an evaluation model. Based on the evaluation set, the first performance gain of the target model on the evaluation set is calculated. This first performance gain is an accuracy evaluation metric, calculated by inputting the training data corresponding to the initial candidate attribute subset in the evaluation set into the target model to obtain the corresponding prediction results. The accuracy evaluation metric for the target model is then calculated, representing the proportion of samples whose predictions match the corresponding true labels out of the total number of samples. The second performance gain of the evaluation model on the evaluation set is also calculated. This is done by inputting the training data corresponding to the initial candidate attribute subset in the evaluation set into the evaluation model to obtain the corresponding prediction results. The accuracy evaluation metric for the evaluation model is then calculated, representing the proportion of samples whose predictions match the corresponding true labels out of the total number of samples. The difference between the second and first performance gains is then calculated to obtain the attribute-level performance gain.

[0058] S205: For any sample, calculate the sample-level performance gain during sample repair based on the target model, evaluation set, remaining training data to be repaired, and pre-repair data. Based on the sample-level performance gain, identify and process key samples to obtain sample-level repair data.

[0059] In step S205, the sample-level performance gain is the performance change value of the target model caused by the sample, and the sample-level repair data is the sample obtained after identifying and processing the key samples, i.e., the sample-level repair data.

[0060] In this embodiment, for any sample, the sample-level performance gain during sample repair is calculated based on the target model, evaluation set, remaining training data to be repaired, and pre-repair data. The sample-level performance gain during sample repair can be calculated using an approximate evaluation method based on changes in model parameters.

[0061] Based on the sample-level performance gain, key samples are identified and processed to obtain sample-level repair data. The sample-level performance gains can be sorted, and the top K samples corresponding to the highest performance gains are extracted as sample-level repair data. Other methods can also be used, and this embodiment does not limit them.

[0062] Optionally, based on the target model, evaluation set, remaining training data to be repaired, and pre-repaired data, the sample-level performance gain during sample repair is calculated, including: Obtain the training parameters of the target model on the training data to be repaired. Calculate the model gradient of the training parameters based on the loss function of the target model. The model gradient includes the first gradient of the loss function with respect to the training parameters based on the remaining training data to be repaired and the second gradient of the loss function with respect to the training parameters based on the pre-repaired data. Construct an update model of the model parameters based on the training parameters, the first gradient, and the second gradient; Based on the target model and the update quantity model, construct a virtual model; Based on the evaluation set, the third performance gain of the virtual model on the evaluation set and the fourth performance gain of the target model on the evaluation set are calculated. The difference between the third and fourth performance gains is calculated to obtain the sample-level performance gain.

[0063] In this embodiment, the training parameters of the target model on the training data to be repaired are obtained. Based on the loss function of the target model, the model gradient of the training parameters is calculated. The model gradient includes a first gradient of the loss function with respect to the training parameters based on the remaining training data to be repaired, and a second gradient of the loss function with respect to the training parameters based on the pre-repaired data. An update model for the model parameters is constructed based on the training parameters, the first gradient, and the second gradient. The formula for the update model of the model parameters is as follows: in, and This is a combination of samples before and after conflict repair of the remaining training data to be repaired. This refers to the pre-repaired data in the sample pair, i.e., the data after conflict repair for sample t. This refers to the remaining training data to be repaired in the sample pair, i.e., the data of sample t before conflict repair. That is, the training parameters of the target model on the training data to be repaired. Let the loss function be the target model. Let t be the loss value of the sample under the target model. For training parameters Partial derivatives under minimal perturbation ε This is the first gradient of the loss function with respect to the training parameters based on the remaining training data to be repaired. This is the second gradient of the loss function with respect to the training parameters based on the pre-repaired data. The loss value is the inverse of the Hessian matrix at parameter θ. To achieve an efficient approximation, a method combining the Hessian-Vector Product and conjugate gradient iteration is used to solve the problem. , The change in model parameters, = - , The amount of time the model parameters are updated, i.e. Approximate value.

[0064] Based on the target model and the update quantity model, a virtual model is constructed, wherein the formula of the virtual model is as follows: in, A virtual model, that is, a target model The copy is used to accumulate and update parameters. For the target model, For updating the quantity model.

[0065] Based on the evaluation set, the third performance gain of the virtual model and the fourth performance gain of the target model on the evaluation set are calculated. The third performance gain represents the accuracy of the virtual model on the evaluation set, and the fourth performance gain represents the accuracy of the target model on the evaluation set. The difference between the third and fourth performance gains is calculated to obtain the sample-level performance gain.

[0066] Optionally, based on the sample-level performance gain, key samples are identified and processed to obtain sample-level repair data, including: In the evaluation set, the top k training data corresponding to the values ​​of sample-level performance gain are determined as sample-level repair data.

[0067] In this embodiment, the sample-level performance gain values ​​are sorted from largest to smallest in the evaluation set, and the training data corresponding to the first k sample-level performance gain values ​​are the sample-level repair data.

[0068] S206: Perform conflict repair on attribute-level repair data and sample-level repair data to obtain the repaired data.

[0069] In step S206, conflict resolution involves processing contradictory data using a preset conflict resolution strategy.

[0070] In this embodiment, when there are inconsistencies in the repair instructions for the same training data between the attribute-level repair data and the sample-level repair data, or when there are different repair requirements for the same attribute of the same training data, it is necessary to process these contradictory data using a preset conflict repair algorithm to ensure that the final repaired data has consistency and accuracy. The preset conflict repair algorithm is a known conflict repair algorithm, and this embodiment does not limit its implementation.

[0071] In another embodiment, the training data to be repaired consists of six relational datasets: Adult, Nursery, Bank, Default, German, and RoadSafety. The target models are Logistic Regression (LR), Multilayer Perceptron (MLP), and the feature-labeled Transformer model FT-Transformer, respectively. After conflict repair using the method described in this application on the training data to be repaired, the target models are trained. The results show that the method described in this application has significant advantages in model performance recovery. The overall average relative accuracy improvement reaches 40.5%, with the highest improvement of 54.8% on the Logistic Regression model and a 24.2% improvement on the FT-Transformer model. Even with the same repair budget, compared to the full-scale cleaning scheme, the method described in this application still outperforms by approximately 12.8% despite a reduced repair scope. These results indicate that the method described in this application can effectively locate the conflict sources that have the greatest impact on model performance, making subsequent repair operations more targeted and thus significantly improving the model's prediction performance.

[0072] Under the same repair cost conditions, embedding the proposed method into existing conflict repair workflows, including typical conflict repair methods such as ROCK, RB (a detection-repair workflow based on Raha and Baran), HoloClean, RT (a detection-repair workflow based on Raha and the pre-trained language model T5), and Mode (a repair method based on attribute mode filling), further improves overall performance by approximately 15%–25%. Experiments show that prioritizing the repair of key attributes and key samples identified by the proposed method can significantly reduce invalid repair operations, and the model accuracy curve improves rapidly and stabilizes in the early stages of repair iterations. Furthermore, comparative results under different noise injection methods (including Self-injector, DeepFool, and Pattern injector) demonstrate that the proposed method exhibits good stability and adaptability to different conflict types. Even when data conflicts are generated by complex patterns (Pattern injector), it still maintains an average performance improvement of over 20%, verifying the robustness of the identified key objects under different data structures and conflict forms.

[0073] The key attribute and key sample identification process of this invention is completed in one go under fixed model conditions, without the need for repeated training. Experimental results show that the total execution time of this method is approximately 451.6 seconds (on a dataset containing 13K samples), which is more than 60 times more efficient than mainstream methods on average. The algorithm converges and outputs stable results within 7 rounds of evaluation iterations, and can directly generate a list of key attributes and samples that can be used in subsequent repair modules. These results demonstrate that this invention maintains high accuracy while being lightweight and highly scalable, and can be directly deployed and used in practical relational data repair systems.

[0074] In summary, the experimental results fully demonstrate that the key attribute and key sample identification method proposed in this invention can significantly improve the accuracy, efficiency and stability of training data conflict repair without increasing the computational burden, and has clear engineering value and practical application prospects.

[0075] This application identifies key attributes in the training data to be repaired, and evaluates the performance gain of different attribute combinations after repair without retraining the model. It also determines which specific attributes should be prioritized for repair. Furthermore, by identifying key samples in the training data to be repaired, it determines which specific training samples should be prioritized for repair based on the impact of sample repair on model parameters. This allows the repair process to focus on the most influential fields and samples, rather than performing comprehensive repair on all attributes and all samples, avoiding numerous low-return modifications. This makes the repair process more targeted and selective, reducing the cost of repairing training data.

[0076] Please see Figure 3 , Figure 3 This is a schematic diagram of a conflict resolution device for relational training data according to an embodiment of this application. This conflict resolution device for relational training data corresponds one-to-one with the conflict resolution method for relational training data in the above embodiments. Please refer to [link / reference] for details. Figure 2 as well as Figure 2 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 3 The conflict repair device 30 for relational training data includes: an acquisition module 31, a filtering module 32, a pre-repair module 33, a first identification module 34, a second identification module 35, and a repair module 36.

[0077] The acquisition module 31 is used to acquire the training data to be repaired and the target model. The training data to be repaired is used to train the target model. The training data to be repaired includes multiple attributes and multiple samples.

[0078] The filtering module 32 is used to obtain the error information of the target model on the training data to be repaired, and to filter the training data to be repaired based on the error information to obtain the evaluation set.

[0079] The pre-repair module 33 is used to determine the remaining training data to be repaired based on the training data to be repaired and the evaluation set, and to perform conflict repair on the remaining training data to be repaired to obtain pre-repaired data.

[0080] The first identification module 34 is used to acquire multiple candidate attribute subsets. For any candidate attribute subset, it calculates the attribute-level performance gain when repairing the candidate attribute subset based on the target model, evaluation set and pre-repair data. It traverses all candidate attribute subsets to obtain the attribute-level performance gain of each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, it identifies and processes key attributes to obtain attribute-level repair data.

[0081] The second identification module 35 is used to calculate the sample-level performance gain for any sample during sample repair based on the target model, evaluation set, remaining training data to be repaired, and pre-repair data. Based on the sample-level performance gain, key samples are identified and processed to obtain sample-level repair data.

[0082] Repair module 36 is used to resolve conflicts between attribute-level repair data and sample-level repair data to obtain the repaired data.

[0083] Optionally, the above-mentioned filtering module 32 includes: The first acquisition unit is used to acquire the predicted category of the training data to be repaired, and to classify the training data to be repaired according to the predicted category, so as to obtain the category training data to be repaired for each category.

[0084] The first processing unit is used to obtain the attribute values ​​of the category training data to be repaired for any category, and to perform hierarchical processing on the category training data to be repaired according to the attribute values ​​to obtain the hierarchical training data to be repaired for each layer.

[0085] The filtering unit is used to filter the training data to be repaired in each layer based on the error information, so as to obtain the set of training data to be repaired after filtering in each layer. By traversing all layers, the set of training data to be repaired after filtering in all layers is obtained, and the set of training data to be repaired after filtering in all layers is determined as the evaluation set.

[0086] Optionally, the above filtering unit includes: The filtering subunit is used to perform initial filtering of the training data to be repaired based on prediction confidence and output stability, and obtain the initial filtering results.

[0087] The sorting unit is used to sort the initial screening results from largest to smallest based on the sample loss value, and to determine the initial screening results that are sorted within a preset range as the evaluation set.

[0088] Optionally, the first identification module 34 includes: The second acquisition unit is used to acquire multiple initial candidate attribute subsets in the current round. For any initial candidate attribute subset, the unit calculates the initial attribute-level performance gain when the initial candidate attribute subset is repaired based on the target model, evaluation set and pre-repair data. The unit then iterates through all initial candidate attribute subsets to obtain the attribute-level performance gain for each initial candidate attribute subset.

[0089] The selection unit is used to select the first m corresponding initial candidate attribute subsets from the attribute-level performance gain values ​​as the seed candidate attribute subset, where m is an integer greater than zero.

[0090] The second processing unit is used to perform crossover and mutation processing on the seed candidate attribute subset to obtain a new candidate attribute subset. The new candidate attribute subset is used as multiple initial candidate attribute subsets for the next round until the preset round is reached, and the final new candidate attribute subset is obtained. The final new candidate attribute subset is determined as multiple candidate attribute subsets.

[0091] Optionally, the first identification module 34 includes: The first determining unit is used to determine the set of pre-repair attribute data in the pre-repair data of the initial candidate attribute subset.

[0092] The training unit is used to acquire a preset lightweight model, train the lightweight model using a pre-fixed attribute dataset, and obtain a trained lightweight model.

[0093] The first computational unit is used to calculate the residual of the target model on the pre-repaired attribute data set based on the pre-repaired attribute data set, and to fit the residual using a trained lightweight model to obtain the residual fitting model.

[0094] The first building unit is used to add the residual fitting model to the target model to build the evaluation model.

[0095] The second calculation unit is used to calculate the first performance gain of the target model on the evaluation set and the second performance gain of the evaluation model on the evaluation set, based on the evaluation set.

[0096] The third calculation unit is used to calculate the difference between the second performance gain and the first performance gain to obtain the attribute-level performance gain.

[0097] Optionally, the second identification module 35 includes: The second acquisition unit is used to acquire the training parameters of the target model on the training data to be repaired, and calculate the model gradient of the training parameters according to the loss function of the target model. The model gradient includes the first gradient of the loss function with respect to the training parameters based on the remaining training data to be repaired and the second gradient of the loss function with respect to the training parameters based on the pre-repaired data. The second building unit is used to build an update model of the model parameters based on the training parameters, the first gradient, and the second gradient.

[0098] The third building unit is used to construct a virtual model based on the target model and the update quantity model.

[0099] The fourth calculation unit is used to calculate the third performance gain of the virtual model on the evaluation set and the fourth performance gain of the target model on the evaluation set, based on the evaluation set.

[0100] The fifth calculation unit is used to calculate the difference between the third performance gain and the fourth performance gain to obtain the sample-level performance gain.

[0101] Optionally, the second identification module 35 further includes: The fourth determination unit is used to determine the top k corresponding training data from the sample-level performance gain values ​​in the evaluation set as sample-level repair data.

[0102] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0103] Figure 4 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. For example... Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in the conflict repair method embodiments for any of the relational training data described above.

[0104] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0105] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0106] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of the computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0108] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0109] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0110] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A conflict resolution method for relational training data, characterized in that, The conflict resolution method includes: Obtain training data to be repaired and a target model. The training data to be repaired is used to train the target model. The training data to be repaired includes multiple attributes and multiple samples. Obtain the error information of the target model on the training data to be repaired, and filter the training data to be repaired based on the error information to obtain an evaluation set; Based on the training data to be repaired and the evaluation set, the remaining training data to be repaired is determined, and conflict repair is performed on the remaining training data to be repaired to obtain pre-repaired data. Multiple candidate attribute subsets are obtained. For any candidate attribute subset, the attribute-level performance gain when the candidate attribute subset is repaired is calculated based on the target model, the evaluation set and the pre-repair data. All candidate attribute subsets are traversed to obtain the attribute-level performance gain of each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, key attributes are identified and processed to obtain attribute-level repair data. For any sample, the sample-level performance gain during sample repair is calculated based on the target model, the evaluation set, the remaining training data to be repaired, and the pre-repair data. Based on the sample-level performance gain, key samples are identified and processed to obtain sample-level repair data. The attribute-level repair data and the sample-level repair data are conflict-corrected to obtain the repaired data.

2. The conflict resolution method as described in claim 1, characterized in that, The step of filtering the training data to be repaired based on the error information to obtain an evaluation set includes: Obtain the predicted category of the training data to be repaired, and classify the training data to be repaired according to the predicted category to obtain the category training data to be repaired for each category. For any category, obtain the attribute values ​​of the category training data to be repaired corresponding to the category, and perform stratification processing on the category training data to be repaired according to the attribute values ​​to obtain the stratified training data to be repaired for each layer. For any layer, the training data to be repaired in the layer is filtered according to the error information to obtain the set of training data to be repaired after filtering for each layer. All layers are traversed to obtain the set of training data to be repaired after filtering for all layers. The set of training data to be repaired after filtering for all layers is determined as the evaluation set.

3. The conflict repair method as described in claim 2, characterized in that, The error information includes sample loss value, prediction confidence, and output stability; The step of filtering the hierarchical training data to be repaired based on the error information to obtain a set of training data to be repaired after each layer of filtering includes: Based on the prediction confidence and the output stability, the training data to be repaired is initially screened to obtain the initial screening results; The initial screening results are sorted from largest to smallest based on their sample loss values, and the initial screening results sorted within a preset range are determined as the evaluation set.

4. The conflict resolution method as described in claim 1, characterized in that, The process of obtaining multiple subsets of candidate attributes includes: Obtain multiple initial candidate attribute subsets for the current round. For any initial candidate attribute subset, calculate the initial attribute-level performance gain when repairing the initial candidate attribute subset based on the target model, the evaluation set, and the pre-repair data. Traverse all initial candidate attribute subsets to obtain the attribute-level performance gain for each initial candidate attribute subset. The first m corresponding initial candidate attribute subsets from the attribute-level performance gain values ​​are selected as the seed candidate attribute subset, where m is an integer greater than zero; The seed candidate attribute subset is subjected to crossover and mutation processing to obtain a new candidate attribute subset. The new candidate attribute subset is used as multiple initial candidate attribute subsets for the next round until the round reaches a preset number of rounds, to obtain the final new candidate attribute subset. The final new candidate attribute subset is then determined as multiple candidate attribute subsets.

5. The conflict resolution method as described in claim 4, characterized in that, The initial attribute-level performance gain for repairing the initial candidate attribute subset is calculated based on the target model, the evaluation set, and the pre-repair data. include: Determine the initial candidate attribute subset as the pre-repair attribute data set in the pre-repair data; Obtain a preset lightweight model, and train the lightweight model using the pre-repaired attribute data set to obtain a trained lightweight model; Based on the pre-repair attribute data set, calculate the residual of the target model on the pre-repair attribute data set, and fit the residual using the trained lightweight model to obtain the residual fitting model; The residual fitting model is added to the target model to construct the evaluation model; Based on the evaluation set, the first performance gain of the target model on the evaluation set and the second performance gain of the evaluation model on the evaluation set are calculated; The difference between the second performance gain and the first performance gain is calculated to obtain the attribute-level performance gain.

6. The conflict resolution method as described in claim 1, characterized in that, The step of calculating the sample-level performance gain during sample repair based on the target model, the evaluation set, the remaining training data to be repaired, and the pre-repaired data includes: The target model's training parameters on the training data to be repaired are obtained, and the model gradient of the training parameters is calculated based on the loss function of the target model. The model gradient includes a first gradient of the loss function with respect to the training parameters based on the remaining training data to be repaired and a second gradient of the loss function with respect to the training parameters based on the pre-repaired data. The update model of the model parameters is constructed based on the training parameters, the first gradient, and the second gradient; Based on the target model and the update quantity model, construct a virtual model; Based on the evaluation set, the third performance gain of the virtual model on the evaluation set and the fourth performance gain of the target model on the evaluation set are calculated; The difference between the third performance gain and the fourth performance gain is calculated to obtain the sample-level performance gain.

7. The conflict resolution method as described in claim 1, characterized in that, The process of identifying and processing key samples based on the sample-level performance gain to obtain sample-level repair data includes: In the evaluation set, the first k corresponding training data values ​​of the sample-level performance gain are determined as sample-level repair data.

8. A conflict repair device for relational training data, characterized in that, The conflict repair device includes: The acquisition module is used to acquire the training data to be repaired and the target model. The training data to be repaired is used to train the target model. The training data to be repaired includes multiple attributes and multiple samples. A filtering module is used to obtain error information of the target model on the training data to be repaired, and to filter the training data to be repaired based on the error information to obtain an evaluation set. The pre-repair module is used to determine the remaining training data to be repaired based on the training data to be repaired and the evaluation set, and to perform conflict repair on the remaining training data to be repaired to obtain pre-repaired data. The first identification module is used to acquire multiple candidate attribute subsets. For any candidate attribute subset, the module calculates the attribute-level performance gain when the candidate attribute subset is repaired based on the target model, the evaluation set, and the pre-repair data. The module iterates through all candidate attribute subsets to obtain the attribute-level performance gain of each candidate attribute subset. Based on the attribute-level performance gain of each candidate attribute subset, the module identifies and processes key attributes to obtain attribute-level repair data. The second identification module is used to calculate the sample-level performance gain when repairing the sample for any sample based on the target model, the evaluation set, the remaining training data to be repaired, and the pre-repair data, and to identify key samples based on the sample-level performance gain to obtain sample-level repair data. The repair module is used to resolve conflicts between the attribute-level repair data and the sample-level repair data to obtain the repaired data.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the conflict resolution method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the conflict resolution method as described in any one of claims 1 to 7.

Images