Method and system for iteratively optimizing a used car valuation model based on human evaluation data

Abstract

The present application relates to a kind of method and system for iteratively optimizing used car valuation model based on artificial evaluation data, and initial committee comprising multiple base valuation models is trained using labeled data.In each iteration, the prediction divergence of unlabeled samples and the feature space data sparsity are calculated comprehensively, a dynamic hybrid selection index is generated, and target samples are selected accordingly.For the selected target samples, search for labeled data within their feature neighborhood, and assign local expert weights based on the historical performance of each base model in the neighborhood.Using the weights, the prediction values of each base model are weighted and fused to generate a fused valuation as a pseudo-label.The pseudo-labeled samples are expanded to the training set and the committee model is retrained.When the number of iterations or overall performance meets the preset conditions, the committee structure optimization is started, and the iteration closed loop is completed.

Description

Technical Field

[0001] This application belongs to the field of model optimization, and in particular relates to a method and system for iteratively optimizing a used car valuation model based on manually evaluated data. Background Technology

[0002] With the rapid development of the used car market and the increasing online nature of transactions, vehicle valuation has become a crucial aspect of promoting market circulation and ensuring fair transactions. To replace assessment methods relying on human experience, machine learning and deep learning models are widely used in used car valuation tasks. These models achieve automated and standardized valuation by learning from historical transaction data. In practical applications, obtaining accurate used car transaction price tags is costly and time-consuming. Furthermore, the data distribution itself suffers from severe imbalance and sparsity, with a large number of rare models or vehicles in special conditions missing samples, resulting in weak generalization ability of the models on this data. To address the data labeling dilemma, active learning and semi-supervised learning techniques can be employed. Among them, committee-based active learning methods identify the most informative samples by measuring the divergence of predictions from multiple base models, representing an effective uncertainty sampling strategy.

[0003] While combining committee methods with active learning and semi-supervised learning can improve the performance of used car valuation models, relying solely on divergence may lead to oversampling in certain difficult regions, neglecting to explore sparse regions in the data space where the model's knowledge is limited. Conversely, focusing solely on data sparsity may result in selecting outliers that contribute little to improving the model's core decision-making capabilities, failing to balance conflict resolution with the exploration of the unknown. Furthermore, the common strategy of simply averaging or voting on the predictions of committee members is problematic because each base model has varying expertise across different feature subspaces. When a sample to be labeled is located in a specific data region, some base models may be experts in that area, while others may be novices. Fusion of these models introduces noise, contaminates the quality of pseudo-labels, and misleads subsequent model training. Moreover, during the iterative evolution of the model, updating only the overall parameters of the committee may lead to a decline in committee diversity or allow some poorly performing base models to become bottlenecks, continuously dragging down overall performance and causing model optimization to stagnate. Summary of the Invention

[0004] This invention proposes a method and system for iteratively optimizing a used car valuation model based on manually evaluated data, which is used to solve the problem of inaccurate valuation caused by insufficient performance of the valuation model in the prior art.

[0005] In the first aspect, this invention proposes a method for iteratively optimizing a used car valuation model based on manually assessed data, comprising the following steps: Using an initial labeled used car dataset, an initial committee containing multiple base estimation models is trained and obtained; in each iteration cycle, for each data sample in the dataset to be labeled, the divergence of the predicted values ​​is quantified using each base estimation model in the committee, and the data sparsity of the samples is calculated in the feature space of the dataset to be labeled. Based on the aforementioned divergence and data sparsity, a dynamic hybrid selection index is generated. According to this index, a batch of target data samples is selected from the dataset to be labeled. For any target data sample, labeled samples are retrieved within its feature space neighborhood. Based on the historical prediction performance of each base estimation model on the labeled samples within that neighborhood, local expert weights are assigned to each base estimation model. Using these local expert weights, the predicted values ​​of each base estimation model for the target data sample are weighted and fused to generate a fused estimate of the target data sample as a pseudo-label. The batch of target data samples with fused valuations are added to the training set to obtain an updated training set, and the updated training set is used to retrain each base valuation model in the committee. After completing a preset number of iterations or when the overall performance index of the committee meets the preset conditions, the committee structure optimization is initiated to complete one iteration optimization.

[0006] In the second aspect, this invention proposes a system for iteratively optimizing a used car valuation model based on manually assessed data, the system comprising the following modules: The computation module is used to train and obtain an initial committee containing multiple base estimation models using the initial labeled used car dataset; in each iteration cycle, for each data sample in the dataset to be labeled, the basis estimation models in the committee are used to quantify the degree of divergence of the predicted values, and the data sparsity of the samples is calculated in the feature space of the dataset to be labeled. An evaluation module is used to generate a dynamic hybrid selection index based on the divergence degree and data sparsity; according to the dynamic hybrid selection index, a batch of target data samples are selected from the dataset to be labeled; for any target data sample, labeled samples are retrieved in the neighborhood of its feature space; and local expert weights are assigned to each base estimation model based on the historical prediction performance of each base estimation model on the labeled samples in the neighborhood; using the local expert weights, the predicted values ​​of each base estimation model for the target data sample are weighted and fused to generate a fused estimate of the target data sample as a pseudo-label. The iterative optimization module adds the batch target data samples with fused valuation to the training set to obtain an updated training set, and uses the updated training set to retrain each base valuation model in the committee; after completing a preset number of iteration cycles or when the overall performance index of the committee meets the preset conditions, the committee structure optimization is initiated, thereby completing one iterative optimization.

[0007] Compared with the prior art, the present invention has at least the following advantages: 1) This invention constructs a sample selection mechanism that takes into account both model prediction divergence and data sparsity. It can prioritize the selection of data that has the greatest value for improving the model, thereby solving the core prediction conflict of the model and exploring the sparse data region, thus improving the iteration efficiency and the utilization value of training data.

[0008] 2) By assigning local expert weights to the base estimation model in the neighborhood of the target sample and performing weighted fusion to generate pseudo-labels, the accuracy and reliability of the labels of the newly added training samples are ensured, and the dependence on expensive manual annotation is reduced.

[0009] 3) By introducing a committee structure optimization process, poorly performing baseline estimation models are identified and replaced regularly, maintaining the diversity and overall predictive ability of the committee and preventing model performance from stagnating or deteriorating during the iteration process.

[0010] This invention enables continuous optimization of model performance at a lower cost and with higher efficiency, resulting in a used car valuation model that is more accurate in prediction, has stronger generalization ability, and is more robust. Attached Figure Description

[0011] Figure 1 A flowchart of the first embodiment; Figure 2 This is a schematic diagram illustrating local error and expert weights. Figure 3 A comparison chart of the overall performance evaluation of the models; Figure 4 A comparison chart of prediction errors for different schemes. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] See Figure 1As shown in the flowchart, in the first embodiment, the present invention proposes a method for iteratively optimizing a used car valuation model based on manually assessed data, including the following steps: Step 1: Using the initial labeled used car dataset, train and obtain an initial committee containing multiple base estimation models; in each iteration cycle, for each data sample in the dataset to be labeled, use each base estimation model in the committee to quantify the divergence of the predicted values, and calculate the data sparsity of the samples in the feature space of the dataset to be labeled. The initial labeled used car dataset is preprocessed. Data is read using the pandas library, and missing values ​​for numerical features are filled with the mean or median and standardized. Categorical features are one-hot encoded. A committee containing M base estimation models is constructed; for example, three models are selected: lightgbm.LGBMRegressor, xgboost.XGBRegressor, and sklearn.ensemble.RandomForestRegressor. To ensure initial diversity, different random seed parameters, such as random_state, are set for each model. The fit method of each model is called to train the model on the complete initial labeled dataset, resulting in the initialized committee, which consists of M trained model objects.

[0014] For any data sample x in the dataset to be labeled, the predict method of the M basis estimation models in the committee is called to obtain M predicted estimates, denoted as p1, p2, ..., pM. The variance of the M predicted values ​​is calculated, and the variance is used as the prediction divergence of sample x. Data sparsity is calculated by constructing a k-nearest neighbor search structure based on the feature vectors of all samples in the dataset to be labeled. For sample x, the structure is used to query its k nearest neighbor samples in the feature space, for example, k is set to 5. The average Euclidean distance between sample x and its k nearest neighbor samples is calculated, and this average distance is used as the data sparsity of sample x.

[0015] Step 2: Based on the divergence and data sparsity, a dynamic hybrid selection index is generated. According to the dynamic hybrid selection index, a batch of target data samples is selected from the dataset to be labeled. For any target data sample, labeled samples are retrieved within its feature space neighborhood. Based on the historical prediction performance of each base estimation model on the labeled samples within the neighborhood, local expert weights are assigned to each base estimation model. Using these local expert weights, the predicted values ​​of each base estimation model for the target data sample are weighted and fused to generate a fused estimate of the target data sample as a pseudo-label. For all samples to be labeled, the calculated dissimilarity and sparsity values ​​are subjected to min-max normalization to scale their range to between 0 and 1, resulting in normalized dissimilarity D and sparsity S. The current iteration period is set to t, and the total iteration period to T. The dynamic weight α is calculated using a linear decay function; in one embodiment, the formula is as follows: The formula for calculating the dynamic mixed selection index H is as follows: In the early stages of iteration, t is close to 0 and α is close to 1, and the index H is mainly determined by the degree of divergence D. As the iteration progresses, t increases, α decreases, and the weight of sparsity S increases accordingly.

[0016] After calculating the value of the dynamic mixture selection index H for each sample in the dataset to be labeled, sort all samples in descending order according to the H value, which can be achieved using the numpy.argsort function; select the B samples with the highest H values ​​from the sorted list, where B is the preset batch size, such as 500. These B samples constitute the target data sample batch to be processed in this iteration.

[0017] Using the feature vectors of all labeled training set samples, construct a KDTree or NearestNeighbors index structure; for any target data sample x, use the index structure to query its k nearest neighbor samples in the labeled dataset to obtain the neighborhood sample set. For the m-th basic estimation model in the committee, traverse each labeled sample in the neighborhood sample set to obtain its true estimation. and historical predictions of model m The prediction error of model m in this neighborhood is calculated using the mean square error function. Local expert weights The calculation formula is ,in, For a very small number, such as Figure 2 As shown, the local expert weights of all base estimation models are normalized, resulting in the final weights of the m-th model. equal Divide by all model weights The sum of all weights ensures that the sum of all weights is 1.

[0018] For the target data sample x, obtain the prediction value of the m-th basic estimation model in the committee. ; predict the value The normalized local expert weights corresponding to the model calculated in the previous step. Multiply; sum the weighted predictions from all base estimation models to obtain the result. , This is the pseudo-label of sample x.

[0019] Step 3: Add the batch target data samples with fused valuation to the training set to obtain an updated training set, and use the updated training set to retrain each base valuation model in the committee; after completing a preset number of iteration cycles or when the overall performance index of the committee meets the preset conditions, start the committee structure optimization to complete one iteration optimization.

[0020] The feature vectors of the selected B target data samples are combined with their corresponding fused estimates (pseudo-labels) to obtain a new labeled dataset. The pandas.concat function is used to merge this new dataset with the current training set to form an updated training set. For each base estimate model in the committee, its fit method is called, using the updated complete training set as input, to retrain or incrementally train it, thereby updating the model parameters.

[0021] In a preferred embodiment, the optimization of the initiation committee structure specifically includes: To select the batch of samples with the highest divergence from the current unlabeled dataset as diagnostic probes, the prediction performance of each base estimation model on the diagnostic probes and its overall performance on the independent validation set are comprehensively evaluated. The worst-performing base estimation model is identified and replaced by a new base estimation model trained on the current updated training set with different random initializations.

[0022] Determine if the iteration termination condition is met, such as the current iteration count reaching a preset maximum value T, or the overall root mean square error of the committee on the independent validation set not decreasing for several consecutive cycles. If the condition is met, from the remaining unlabeled dataset that was never selected as target samples, sort them in descending order of prediction divergence, and select the C samples with the highest divergence as diagnostic probes (e.g., C = 1000). For each baseline estimation model m in the committee, calculate two performance metrics: performance on the diagnostic probes, measured by the mean square error between the predicted value of model m on the probe samples and the average predicted value of all models in the committee, denoted as . The overall performance on the independent validation set is measured by calculating the root mean square error of model m on that validation set, which is denoted as . After normalizing these two metrics, a comprehensive negative review score is calculated through weighted summation. The base evaluation model with the highest negative review score is selected as the replacement model. A new base evaluation model of the same type as the replacement model is initialized, but a new random number is set for its random_state parameter to ensure that its initial weights are different. Using the latest updated training set, the fit method of the new model is called for complete training, and the trained new model replaces the worst-performing old model in the committee.

[0023] In a preferred embodiment, the degree of divergence of the quantified predicted values ​​includes: For any data sample in the dataset to be labeled, use the full base estimation model in the committee to predict it and obtain multiple predicted values; Calculate the variance of the predicted values ​​and use the variance as the degree of divergence of the data samples.

[0024] For any data sample x in a dataset to be labeled, the committee first... Each of the M basis estimation models is used to estimate the value. In one embodiment, the basis estimation model is a feedforward neural network, and the input is a d-dimensional feature vector. The network includes multiple attributes of the data samples, such as numerical features like vehicle age, mileage, and engine displacement, as well as categorical features like brand and model, which have undergone one-hot encoding. All features are normalized. The specific structure includes an input layer matching the feature dimension *d*, two fully connected hidden layers, and an output layer. For example, the number of neurons in the two hidden layers can be set to 64 and 32 respectively. The hidden layer neurons use the Modified Linear Unit (ReLU) as the activation function, expressed as f(z) = max(0,z). The output layer consists of a single neuron using a linear activation function. The output of this network is a scalar. , representing the model's estimate of sample x.

[0025] After all M base estimation models in the committee estimate the sample x, a set of predicted values ​​is generated. For example, a committee composed of 5 base models estimates the value of a certain vehicle to be {10.5, 10.8, 10.2, 11.0, 10.4} ten thousand yuan. The sample variance of this set of predicted values ​​is calculated as the divergence D(x) of the data sample x, and the formula is as follows: ,in It is the average of the predicted values. Taking the aforementioned data as an example, the average value... If the value is 105,800 yuan, then its degree of divergence D(x) is approximately 0.102. The larger the value, the more severe the conflict between models.

[0026] In a preferred embodiment, calculating the data sparsity of the samples includes: For any data sample in the dataset to be labeled, after standardizing or normalizing the sample features, the k-nearest neighbor algorithm is used to find the k nearest data samples in the feature space. Calculate the average Euclidean distance between the data sample and its k nearest neighbors, and use the average Euclidean distance as the data sparsity of the data sample, where k is a preset integer.

[0027] Z-score standardization is applied to the features of the full dataset, including both labeled and unlabeled samples. This involves subtracting the mean from each feature dimension and dividing by its standard deviation to ensure all features are on a uniform scale. For sample x, an efficient k-nearest neighbor search algorithm based on KD-trees or ball trees is used to find the k nearest Euclidean distance samples in the entire feature space or only in the training set. The value of k is typically between 5 and 20, with k being preferred at 10. The arithmetic mean of the Euclidean distances from sample x to its k nearest neighbors is calculated; this average distance is the data sparsity S(x). For example, if the normalized Euclidean distances between sample x's 10 nearest neighbors and itself are {0.12, 0.15, ..., 0.21}, then its data sparsity is the average of these distances, such as 0.172. A larger value indicates sparser data points in the region where sample x is located, suggesting that the model may not learn sufficiently in that region.

[0028] In a preferred embodiment, generating a dynamic hybrid selection metric based on the divergence degree and data sparsity includes: For any data sample in the dataset to be labeled, its divergence and sparsity are normalized to obtain normalized divergence and normalized sparsity, respectively. The dynamic hybrid selection index is calculated by multiplying the normalized divergence by the first weight factor and the normalized sparsity by the second weight factor. The first weight factor decreases as the current iteration number increases, and the second weight factor increases as the current iteration number increases.

[0029] In the entire unlabeled dataset, calculate the set of dissimilarity {D} and the set of sparsity {S} for all samples. Apply the min-max normalization method to both sets to scale the dissimilarity and sparsity values ​​of each sample to the [0,1] interval, obtaining the normalized dissimilarity. and normalized sparsity The dynamic mixing selection index H(x,t) is calculated using the following formula: Where t is the current iteration cycle number, and T is the preset total iteration cycle number, for example, T equals 50. The weight factor is dynamically adjusted; a preferred linear adjustment strategy is: For example, in the 5th iteration, t equals 5 and T equals 50, with weights of... ,index At this point, samples with high divergence are selected first; and in the 40th iteration, when t equals 40, the weights become... ,index At this point, the focus is on exploring regions with sparse data.

[0030] In a preferred embodiment, assigning local expert weights to each base estimation model based on its historical prediction performance on labeled samples within the neighborhood includes: For a target data sample, the k-nearest neighbor algorithm is used to find k labeled neighboring samples in its feature space to form a neighborhood; For any basic estimation model in the committee, calculate its mean absolute error in that neighborhood and normalize it to obtain the normalized mean absolute error; set the local expert weights of the model as the output of a preset function, the input of which is the normalized mean absolute error, and normalize the weights of all models.

[0031] For a target data sample x selected based on a dynamic hybrid selection metric, the k-nearest neighbor algorithm is used in its feature space, where the value of k can be set to 20 to 50, for example, k equals 30, to retrieve the k nearest labeled samples from the current labeled training set, forming its local neighborhood. For the m-th basis valuation model in the committee Calculate its value in its neighborhood. Historical forecast performance, specifically the Mean Absolute Error (MAE), is calculated using the following formula: , among which samples and its labels Belongs to the neighborhood To convert errors into weights, i.e., the smaller the error, the larger the weight, we first... Perform the reciprocal to obtain the initial weights. Similarly It is a very small positive number, such as 1e-8, used to prevent the denominator from being zero. For example, if the local MAE of the three models are {0.5, 1.2, 0.8} ten thousand yuan respectively, then the initial weights... The approximate values ​​are {2.0, 0.833, 1.25}. To make the weight allocation more discriminative and ensure a sum of 1, the Softmax function is used to apply the initial weights to all models. Normalization is performed to obtain local expert weights. The calculation formula is: ,in It is a temperature coefficient, for example, equal to 0.5, used to adjust the smoothness of the weight distribution. The smaller the value, the more concentrated the weight distribution is on the best-performing model.

[0032] In a preferred embodiment, the step of using the local expert weights to weight and fuse the predicted values ​​of each base estimation model for the target data sample to generate a fused estimate of the target data sample as a pseudo-label includes: The fusion estimate of the target data sample is obtained by weighted summation of the predictions of all base estimation models in the committee, where the prediction of each base estimation model is multiplied by its assigned local expert weight.

[0033] Calculate the basic estimation models for the target data sample x. Local expert weights Then, the original predicted values ​​of x for each model are obtained. The final fusion estimate of the sample, i.e., the pseudo-label, is generated by weighted summation. The calculation formula is: For example, if the predicted values ​​of the three models are {10.2, 11.5, 10.8} yuan respectively, and their corresponding local expert weights calculated by Softmax are {0.758, 0.073, 0.169}, then the fusion valuation... The estimated value is 10,000 yuan. This fused valuation, used as a pseudo-label for x, is added to the next round of training data along with x.

[0034] In a preferred embodiment, the comprehensive evaluation of the predictive performance of each base estimation model on the diagnostic probe and its overall performance on an independent validation set, identifying and replacing the worst-performing base estimation model, includes: For each base estimate model in the committee, the average deviation of its predicted values ​​on all diagnostic probes from the committee's average predicted values ​​is calculated, and its average prediction error on the independent validation set is calculated; a comprehensive elimination index is obtained based on the ratio of the absolute value of the average prediction error to the average deviation; the base estimate model with the largest comprehensive elimination index value is identified as the worst-performing model.

[0035] During the committee structure optimization phase, the 1% of samples with the highest divergence from the current unlabeled dataset are selected as the diagnostic probe set. For the m-th base model in the committee Two core metrics are calculated: one is the average deviation on the diagnostic probe. Its value is All diagnostic probes The arithmetic mean of the above, where It is the committee's The first is the average predicted value, which measures the degree of deviation of the model from the consensus on disputed samples; the second is its overall generalization error on the independent validation set V, using the root mean square error, which is denoted as . The square root of the arithmetic mean on the independent validation set V, where For the validation set samples, The corresponding label. The Comprehensive Elimination Index (CEI) identifies the worst-performing model; this index is... It is used to punish poor generalization ability, i.e., high root mean square error, but also lacks a unique perspective. A low-level, blindly following popular models, because this popular consensus may be wrong, such as... Figure 3 The following model is shown for elimination. For example, model A has an RMSE of 0.9, an AD of 0.2, and a CEI of 4.5; model B has an RMSE of 0.5, an AD of 0.8, and a CEI of 0.625. Model A is identified as the worst-performing model due to its high CEI value. This model will be removed from the committee and replaced by a new model trained from scratch on the current latest training set using different random seeds, such as different initial weights for the neural network.

[0036] Using a used car transaction dataset containing 50,000 samples, 500 samples were initially randomly selected as the labeled training set, 5,000 as the fixed validation set, and the remainder as a pool of samples to be labeled. Active learning was performed in 50 rounds, with 100 samples selected from the unlabeled pool for labeling in each round. The committee consisted of five feedforward neural networks with identical structures but different random initializations. The performance evaluation metric was the root mean square error (RMSE) of the model on the fixed validation set, expressed in tens of thousands of yuan.

[0037] The experiment consisted of five groups: the complete scheme proposed in this invention, and four control groups. These included groups with the committee optimization module removed, groups with the local expert weighting module removed and only using divergence as the selection metric, and groups with baseline random sampling. After 50 rounds of active learning, i.e., when the training set was expanded to 5500 samples, the specific data obtained on the validation set for each group are as follows: Complete scheme: Final validation set RMSE was 0.58 million. Control group 1 (no committee optimization): RMSE was 0.61 million. Control group 2 (using dynamic hybrid selection but employing simple average fusion to generate pseudo-labels): RMSE was 0.67 million. Control group 3 (using only divergence selection): RMSE was 0.72 million. Control group 4 (random sampling): RMSE was 0.85 million.

[0038] See Figure 4 The comparison chart of different schemes shows that, compared to the random sampling baseline, the complete scheme reduces the RMSE by 31.8%, demonstrating the overall effectiveness of the active learning framework. Compared to the strategy using only divergence, introducing a dynamic mixing metric based on data sparsity reduces the RMSE from 0.72 million to 0.67 million, indicating that exploring sparse regions in the later stages of learning effectively fills the model's knowledge gaps and improves generalization ability. Furthermore, using local expert weights for pseudo-label generation further reduces the RMSE from 0.67 million to 0.61 million compared to simple averaging fusion, demonstrating that dynamically matching samples to locally optimal models significantly improves pseudo-label quality.

[0039] In a second embodiment, the present invention also proposes a system for iteratively optimizing a used car valuation model based on manually assessed data, the system comprising the following modules: The computation module is used to train and obtain an initial committee containing multiple base estimation models using the initial labeled used car dataset; in each iteration cycle, for each data sample in the dataset to be labeled, the basis estimation models in the committee are used to quantify the degree of divergence of the predicted values, and the data sparsity of the samples is calculated in the feature space of the dataset to be labeled. An evaluation module is used to generate a dynamic hybrid selection index based on the divergence degree and data sparsity; according to the dynamic hybrid selection index, a batch of target data samples are selected from the dataset to be labeled; for any target data sample, labeled samples are retrieved in the neighborhood of its feature space; and local expert weights are assigned to each base estimation model based on the historical prediction performance of each base estimation model on the labeled samples in the neighborhood; using the local expert weights, the predicted values ​​of each base estimation model for the target data sample are weighted and fused to generate a fused estimate of the target data sample as a pseudo-label. The iterative optimization module adds the batch target data samples with fused valuation to the training set to obtain an updated training set, and uses the updated training set to retrain each base valuation model in the committee; after completing a preset number of iteration cycles or when the overall performance index of the committee meets the preset conditions, the committee structure optimization is initiated, thereby completing one iterative optimization.

[0040] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0041] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0042] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for iteratively optimizing a used car valuation model based on manually assessed data, characterized in that, Includes the following steps: Using an initial labeled used car dataset, an initial committee containing multiple base estimation models is trained and obtained; in each iteration cycle, for each data sample in the dataset to be labeled, the divergence of the predicted values ​​is quantified using each base estimation model in the committee, and the data sparsity of the samples is calculated in the feature space of the dataset to be labeled. Based on the aforementioned divergence and data sparsity, a dynamic hybrid selection index is generated. According to this index, a batch of target data samples is selected from the dataset to be labeled. For any target data sample, labeled samples are retrieved within its feature space neighborhood. Based on the historical prediction performance of each base estimation model on the labeled samples within that neighborhood, local expert weights are assigned to each base estimation model. Using these local expert weights, the predicted values ​​of each base estimation model for the target data sample are weighted and fused to generate a fused estimate of the target data sample as a pseudo-label. The batch of target data samples with fused valuations are added to the training set to obtain an updated training set, and the updated training set is used to retrain each base valuation model in the committee. After completing a preset number of iterations or when the overall performance index of the committee meets the preset conditions, the committee structure optimization is initiated to complete one iteration optimization.

2. The method according to claim 1, characterized in that, The degree of divergence of the quantified predicted values ​​includes: For any data sample in the dataset to be labeled, use the full base estimation model in the committee to predict it and obtain multiple predicted values; Calculate the variance of the predicted values ​​and use the variance as the degree of divergence of the data samples.

3. The method according to claim 1, characterized in that, The calculation of data sparsity of the sample includes: For any data sample in the dataset to be labeled, after standardizing or normalizing the sample features, the k-nearest neighbor algorithm is used to find the k nearest data samples in the feature space. Calculate the average Euclidean distance between the data sample and its k nearest neighbors, and use the average Euclidean distance as the data sparsity of the data sample, where k is a preset integer.

4. The method according to claim 1, characterized in that, The generation of dynamic hybrid selection metrics based on the divergence degree and data sparsity includes: For any data sample in the dataset to be labeled, its divergence and sparsity are normalized to obtain normalized divergence and normalized sparsity, respectively. The dynamic hybrid selection index is calculated by multiplying the normalized divergence by the first weight factor and the normalized sparsity by the second weight factor. The first weight factor decreases as the current iteration number increases, and the second weight factor increases as the current iteration number increases.

5. The method according to claim 1, characterized in that, The process of assigning local expert weights to each base estimation model based on its historical prediction performance on labeled samples within the neighborhood includes: For a target data sample, the k-nearest neighbor algorithm is used to find k labeled neighboring samples in its feature space to form a neighborhood; For any basic estimation model in the committee, calculate its mean absolute error in that neighborhood and normalize it to obtain the normalized mean absolute error; set the local expert weights of the model as the output of a preset function, the input of which is the normalized mean absolute error, and normalize the weights of all models.

6. The method according to claim 1, characterized in that, The step of using the local expert weights to weight and fuse the predicted values ​​of each base estimation model for the target data sample to generate a fused estimate of the target data sample as a pseudo-label includes: The fusion estimate of the target data sample is obtained by weighted summation of the predictions of all base estimation models in the committee, where the prediction of each base estimation model is multiplied by its assigned local expert weight.

7. The method according to claim 1, characterized in that, The optimization of the startup committee structure is as follows: To select the batch of samples with the highest divergence from the current unlabeled dataset as diagnostic probes, the prediction performance of each base estimation model on the diagnostic probes and its overall performance on the independent validation set are comprehensively evaluated. The worst-performing base estimation model is identified and replaced by a new base estimation model trained on the current updated training set with different random initializations.

8. The method according to claim 7, characterized in that, The comprehensive evaluation of the predictive performance of each base estimation model on the diagnostic probe and its overall performance on the independent validation set, identifying and replacing the worst-performing base estimation model, includes: For each base estimate model in the committee, the average deviation of its predicted values ​​on all diagnostic probes from the committee's average predicted values ​​is calculated, and its average prediction error on the independent validation set is calculated; a comprehensive elimination index is obtained based on the ratio of the absolute value of the average prediction error to the average deviation; the base estimate model with the largest comprehensive elimination index value is identified as the worst-performing model.

9. A system for iteratively optimizing a used car valuation model based on manually assessed data, characterized in that, The system includes the following modules: The computation module is used to train and obtain an initial committee containing multiple base estimation models using the initial labeled used car dataset; in each iteration cycle, for each data sample in the dataset to be labeled, the basis estimation models in the committee are used to quantify the degree of divergence of the predicted values, and the data sparsity of the samples is calculated in the feature space of the dataset to be labeled. An evaluation module is used to generate a dynamic hybrid selection index based on the divergence degree and data sparsity; according to the dynamic hybrid selection index, a batch of target data samples are selected from the dataset to be labeled; for any target data sample, labeled samples are retrieved in the neighborhood of its feature space; and local expert weights are assigned to each base estimation model based on the historical prediction performance of each base estimation model on the labeled samples in the neighborhood; using the local expert weights, the predicted values ​​of each base estimation model for the target data sample are weighted and fused to generate a fused estimate of the target data sample as a pseudo-label. The iterative optimization module adds the batch target data samples with fused valuation to the training set to obtain an updated training set, and uses the updated training set to retrain each base valuation model in the committee; after completing a preset number of iteration cycles or when the overall performance index of the committee meets the preset conditions, the committee structure optimization is initiated, thereby completing one iterative optimization.

10. The system according to claim 9, characterized in that, The degree of divergence of the quantified predicted values ​​includes: For any data sample in the dataset to be labeled, use the full base estimation model in the committee to predict it and obtain multiple predicted values; Calculate the variance of the predicted values ​​and use the variance as the degree of divergence of the data samples.

Images