An automatic evaluation method for automobile exterior styling design driven by big data

Abstract

A big data-driven automatic evaluation method for automotive exterior styling design is proposed. First, a large-scale multi-view automotive image dataset is created for vehicle angle recognition and grade recognition, and deep neural network methods are used to train the corresponding vehicle angle recognition and grade recognition machines. Next, an automotive styling evaluation dataset is created for exterior styling scoring. This dataset includes multi-view automotive images and corresponding brand, model, and other labeled information, with styling score annotations and semantic evaluation annotations for end users and styling experts respectively. The automotive styling evaluation dataset is then split into corresponding subsets according to vehicle grade and evaluator attributes, and trained using deep learning regression and classification methods to obtain automatic scoring machines and automatic semantic evaluation machines at different levels for automotive exterior styling users and experts. The significant features affecting styling scores and semantic evaluations are visualized, providing stylists with real-time and objective styling design evaluations during the development process.

Description

Technical Field

[0001] This invention relates to the field of big data-driven automatic evaluation of automotive exterior design, and to a method for real-time automatic evaluation of automotive styling. Background Technology

[0002] The quality of a car's styling reflects the stylist's work and is a crucial factor for consumers when purchasing a car. Car enthusiasts and industry professionals also appreciate and judge its aesthetics. A car's exterior design that aligns with popular aesthetics can instantly capture attention and resonate with consumer psychology. Therefore, designing car exteriors that cater to user preferences and needs is extremely important for automakers. Currently, most car brands have competing models in the same segment, and these models often share similar features, powertrains, and reliability. This makes exterior styling a paramount element of differentiation.

[0003] In the face of fierce market competition and the digital and intelligent transformation of the automotive industry, automotive exterior design, as a crucial part of R&D, urgently needs technological innovation. During the R&D process, reliable evaluation models are needed to effectively assess the car styling designed by designers in order to create designs that satisfy consumers. Traditional automotive styling evaluation research largely relies on manual feature definition and extraction. Although automotive styling analysis methods using predefined features have made significant progress, these methods are highly dependent on manual extraction. In the era of big data, machine learning and deep learning are developing rapidly, and data mining and artificial intelligence are widely applied. Machine learning, represented by deep learning, has made tremendous progress, and computer vision is developing particularly rapidly. The advantage of deep learning methods lies in integrating feature extraction, feature selection, classifiers, and regressors, enabling fully automated styling evaluation and minimizing the subjectivity of manually specified features.

[0004] Based on the above reasons, relevant scholars have proposed an automatic scoring method for automotive exterior styling (invention patent, authorization announcement number: CN 112258472 B). This method establishes a reliable user score for automotive exterior styling based on multi-dimensional user attributes and achieves automatic quantitative scoring of automotive exterior styling through big data-driven and deep learning methods. However, users are limited by their professional knowledge and cannot make a good professional evaluation of automotive styling, and the automotive styling score cannot provide a comprehensive reference for stylists. Therefore, this paper adds an expert perspective to the evaluation of automotive exterior styling based on the above research, and extends the output of automotive exterior styling scores to semantic evaluation output. Considering both user and expert evaluation perspectives and different car levels, car brands, and car angles, an automatic evaluation method based on deep learning multi-view automotive images is used to achieve reasonable and automatic evaluation of automotive exterior styling. Summary of the Invention

[0005] This invention provides an automatic evaluation method for automotive exterior styling. First, a large-scale, multi-view automotive image dataset is created for vehicle angle recognition and grade recognition, and this dataset is labeled with brand, model, and angle information. Deep learning methods are used to train network models to obtain corresponding vehicle angle recognition machines and vehicle grade recognition machines. An automotive styling evaluation dataset is created for exterior styling evaluation, with styling score annotations and styling semantic evaluation annotations for end users and styling experts respectively. The automotive styling evaluation dataset is split into corresponding subsets according to vehicle grade and evaluator attributes, and deep learning regression and classification methods are used to train these subsets to obtain automatic scoring machines and automatic semantic evaluation machines at different levels for automotive exterior styling users and experts. The significant features affecting styling scores and semantic evaluations are visualized, providing stylists with real-time and objective styling design evaluations during the development process.

[0006] The technical solution of this invention:

[0007] An automated evaluation method for automotive exterior styling design driven by big data, comprising the following steps:

[0008] Step 1: Create a large-scale multi-view car image dataset for car angle recognition and grade recognition, and annotate relevant information;

[0009] (1.1) Collect and organize multi-view images of car exteriors of different brands and models, with a total sample size of no less than N1 images, covering no less than N2 car brands, and the model year spanning Y1 to Y2. For each car, N3 images are evenly sampled from a horizontal perspective around the perimeter.

[0010] (1.2) Label each of the multi-view car exterior images in step (1.1), including car class, brand, model and car angle. The car class is divided into four levels: sedans, A, B, C and D, and SUVs, compact, mid-size, mid-large and large SUVs. This results in a large-scale multi-view car image dataset, which is then split into corresponding sub-datasets according to car class.

[0011] Step 2: Train the multi-view car image dataset from Step 1 for angle recognition and grade recognition respectively, and obtain the car angle recognition machine and car grade recognition machine accordingly.

[0012] (2.1) The subset of data split in step 1 is split into corresponding training and validation sets according to the ratio K. A deep learning regression network is used to train the car angle recognition machine, and a deep learning classification network is used to train the car grade recognition machine.

[0013] (2.2) Use the car angle recognition machine and car grade recognition machine obtained in step (2.1) to quickly and accurately identify the input car exterior image and predict the angle and grade of the input car image;

[0014] Step 3: Create a car exterior styling scoring dataset containing user and expert levels for training the automatic car exterior styling scoring machine, and a car exterior styling semantic evaluation dataset containing user and expert levels for training the automatic car exterior styling semantic evaluation machine. Process the multi-view car image dataset from Step 1, and divide it into two groups: end users and styling experts. Perform styling scoring annotation and styling semantic evaluation annotation on the multi-view car image dataset respectively, to obtain the user car styling evaluation dataset containing the car exterior styling user scoring dataset and the car exterior styling user semantic evaluation dataset, and the expert car styling evaluation dataset containing the car exterior styling expert scoring dataset and the car exterior styling expert semantic evaluation dataset.

[0015] (3.1) Collect and organize user rating data corresponding to the multi-view car image dataset in step 1, clean the user rating data, analyze and quantify the multi-dimensional attributes of users, assign different rating weights according to different user attributes, and obtain the comprehensive user rating of car exterior styling; use the comprehensive user rating to annotate the car images in the multi-view car image dataset with corresponding user ratings to obtain the car exterior styling user rating dataset.

[0016] (3.2) Collect and organize user semantic evaluation data corresponding to the multi-view car image dataset in step 1, extract and analyze car styling semantic evaluation keywords from the user semantic evaluation data, and then perform one-to-one user semantic evaluation annotation on the car images in the multi-view car image dataset to obtain the car exterior styling user semantic evaluation dataset.

[0017] (3.3) Collect and organize the expert rating data corresponding to the multi-view car image dataset in step 1. Analyze the reliability of the expert evaluation based on the experts' years of experience and the number of car models evaluated. Clean the expert rating data. Use the expert rating data to annotate the car images in the multi-view car image dataset with corresponding expert ratings to obtain the car exterior styling expert rating dataset.

[0018] (3.4) Collect and organize expert semantic evaluation data corresponding to the multi-view car image dataset in step 1, extract and analyze car styling semantic evaluation keywords from the expert semantic evaluation data, and then perform one-to-one expert semantic evaluation annotation on the car images in the multi-view car image dataset to obtain the car exterior styling expert semantic evaluation dataset.

[0019] Step 4: Split the car exterior styling rating dataset created in Step 3 into corresponding sub-datasets according to car grade and evaluator attributes. Use deep learning regression methods to train the user and expert levels respectively to obtain the car exterior styling user automatic rating machine and the car exterior styling expert automatic rating machine.

[0020] (4.1) Classify the user rating dataset of car exterior styling in step (3.1) according to the car level classification method in step (1.2) to obtain the user rating subset of car exterior styling of X-level models, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning regression method to establish the mapping relationship between styling scores and car exterior styling features in the subset, train deep learning network models respectively, and obtain the automatic user rating machine of car exterior styling of X-level models corresponding to the subset;

[0021] (4.2) Classify the car exterior styling expert rating dataset in step (3.3) according to the car level classification method in step (1.2) to obtain the X-level car exterior styling user rating subset, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning regression method to establish the mapping relationship between styling rating and car exterior styling features in the subset, train deep learning network models respectively, and obtain the X-level car exterior styling expert automatic rating machine corresponding to the subset;

[0022] Step 5: Split the car exterior styling semantic evaluation dataset created in Step 3 into corresponding sub-datasets according to car grade and evaluator attributes. Use deep learning multi-label classification method to train the user and expert levels respectively to obtain the car exterior styling user automatic semantic evaluation machine and the car exterior styling expert automatic semantic evaluation machine.

[0023] (5.1) Classify the user semantic evaluation dataset of car exterior styling in step (3.2) according to the car level classification method in step (1.2) to obtain the user rating subset of X-level car exterior styling, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning multi-label classification method to establish the mapping relationship between styling semantic evaluation and car exterior styling features in the subset, train deep learning multi-label classification models respectively, and obtain the automatic semantic evaluation machine of X-level car exterior styling users corresponding to the subset;

[0024] (5.2) Classify the semantic evaluation dataset of car exterior styling experts in step (3.4) according to the car level classification method in step (1.2) to obtain the user rating subset of X-level car exterior styling, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning multi-label classification method to establish the mapping relationship between styling semantic evaluation and car exterior styling features in the subset, train deep learning multi-label classification models respectively, and obtain the automatic semantic evaluation machine of X-level car exterior styling experts corresponding to the subset;

[0025] Step 6: Use deep learning model visualization methods to visualize the salient features of the automatic car exterior styling scoring machine and the automatic car exterior styling semantic evaluation machine, respectively.

[0026] (6.1) A deep learning model visualization method (gradient-weighted activation mapping method) is used to visualize the salient features of car styling. This method uses the gradient information flowing into the convolutional layers of the CNN to understand the importance of each neuron in the prediction target. For the final output c (c represents the category or regression value), the weight formula of the Kth neuron in the last convolutional layer is as follows:

[0027]

[0028] in, is the sensitivity of c relative to the k-th channel of the output feature map of the last convolutional layer, Z is the number of pixels in the feature map, i and j represent the indices of the width and height dimensions, respectively, and y c This represents the probability of c in the output of the last activation function. This is the feature map output by the last convolutional layer, where k is the index of the channel dimension of the feature map. This involves taking the partial derivative of the probability value with respect to all pixels in the output feature map of the k-th neuron in the last layer; then... The feature maps output from the last convolutional layer are weighted and linearly combined, and then processed by the ReLU activation function to filter out negative values ​​obtained by weighting at a certain position on feature map A, retaining only the outputs positively correlated with the predicted value. The overall formula is as follows:

[0029]

[0030] Where L is a two-dimensional category activation heatmap, A k It is the feature map output by the last convolutional layer. By superimposing the heatmap on the original image, a visualization effect of significant features is presented.

[0031] (6.2) Using the deep learning model visualization method introduced in step (6.1), visualize the salient features of the car exterior styling user automatic rating machine, user automatic semantic evaluation machine, expert automatic rating machine, and expert automatic semantic evaluation machine obtained in steps (4.1), (4.2), (5.1), and (5.5) respectively, and locate the car exterior styling features that users and experts are interested in in the form of heat maps.

[0032] Step 7: For the input exterior design renderings, use the automatic scoring machine and automatic semantic evaluation machine for automotive exterior styling obtained in Steps 2 and 4-6 to provide designers with real-time objective exterior styling scores, semantic evaluations, and corresponding visual information of significant features from both user and styling expert perspectives.

[0033] (7.1) For the input car exterior design rendering, the car exterior angle recognition machine and car exterior level recognition machine obtained in step (2.1) are used to determine the car model angle and car model level;

[0034] (7.2) After determining the angle and level of the vehicle model in step (7.1), the user score is obtained by the user automatic scoring machine corresponding in step (4.1), and the expert score is obtained by the expert automatic scoring machine corresponding in step (5.1). The saliency feature visualization method in step (6.1) is used to output the vehicle exterior styling features that the user automatic scoring machine and the expert automatic scoring machine focus on when scoring.

[0035] (7.3) After determining the angle and level of the vehicle model in step (7.1), the user semantic evaluation is obtained by the user automatic semantic evaluation machine corresponding in step (4.2), and the expert semantic evaluation is obtained by the expert automatic semantic evaluation machine corresponding in step (5.2). The saliency feature visualization method in step (6.1) is used to output the vehicle exterior styling features that the user automatic semantic evaluation machine and the expert automatic semantic evaluation machine pay attention to when performing semantic evaluation of the vehicle exterior styling.

[0036] The beneficial effects of this invention are as follows: This invention provides an automatic evaluation method for automotive exterior styling. Addressing the problem that traditional automotive exterior styling evaluation processes often rely on small sample data and subjective evaluation methods, this invention considers multi-dimensional user attributes using big data to obtain reasonable styling scores. Based on different perspectives, deep learning methods are used to construct a mapping relationship between automotive styling evaluation data and automotive styling features, resulting in an automatic scoring machine and an automatic semantic evaluation machine for automotive exterior styling based on different levels of user and expert input. The significant features affecting styling scores and semantic evaluations are visualized, providing stylists with a reference for subsequent R&D processes.

[0037] (1) Overcome the shortcomings of traditional car styling evaluation methods, such as small sample data and strong subjectivity. Use big data to drive deep learning, consider different user attributes, and obtain reasonable car appearance evaluation feedback, which has certain reference value for designers to grasp user psychology.

[0038] (2) Deep learning methods are used to train models according to car class, and a mapping relationship is established between multi-view images of car styling and car styling scores and semantic evaluations. This avoids reliance on manual feature definition and extraction, and has higher robustness and accuracy. The significant features in the car exterior styling evaluation process are visualized, and the significant feature regions that affect car styling scores and styling semantic evaluations are identified, providing a reference direction for the subsequent R&D process. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the overall structure of the present invention.

[0040] Figure 2 Flowchart of the comprehensive scoring mechanism for automotive exterior styling.

[0041] Figure 3 This is a schematic diagram of the automatic scoring process for automobile exterior styling.

[0042] Figure 4 Flowchart for angle-based scoring

[0043] Figure 5 This is a schematic diagram of the automatic semantic evaluation process for automotive exterior styling.

[0044] Figure 6 This is a flowchart illustrating the process of visualizing deep learning models.

[0045] Figure 7 This is a demonstration diagram of an automatic evaluation system for automotive exterior styling. Detailed Implementation

[0046] To provide a more detailed explanation of the steps of this invention, specific implementation processes are described using accompanying drawings and examples. The examples described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0047] See Figure 1 This invention provides an automatic evaluation method for automotive exterior styling. First, a large-scale, multi-view automotive image dataset is created and relevant information is labeled. A deep learning model is trained using this dataset to obtain automatic scoring machines and automatic semantic evaluation machines at different levels, including those for automotive exterior styling users and experts. A deep learning model visualization method is used to visualize the significant features affecting styling scores and semantic evaluations, providing stylists with real-time, objective styling design evaluations during the development process.

[0048] The aforementioned big data-driven automatic evaluation method for automotive exterior design mainly includes the following steps:

[0049] Step 1 specifically includes the following sub-steps:

[0050] (1.1) Collect and organize multi-view images of car exteriors of different brands and models, with a total sample size of no less than 20,000 images, covering no less than 16 car brands, with a model age span of no less than 5 years, and sampling 30 images of each car evenly around the perimeter from a level view.

[0051] (1.2) Label each of the multi-view car exterior images in step (1.1), including car class, brand, model and car angle. The car class is divided into four levels: sedans, A, B, C and D, and SUVs, compact, mid-size, mid-large and large SUVs. This results in a large-scale multi-view car image dataset, which is then split into corresponding sub-datasets according to car class.

[0052] Step 2 specifically includes the following sub-steps:

[0053] (2.1) The data in the subset of data split in step 1 is split into corresponding training sets and validation sets according to the ratio K = 7 / 3. A deep learning regression network is used to train the car appearance angle recognition machine, and a deep learning classification network is used to train the car appearance level recognition machine.

[0054] (2.2) Use the car exterior angle recognition machine and car exterior grade recognition machine obtained in step (2.1) to quickly and accurately identify the input car exterior image and predict the angle and grade of the input car exterior image;

[0055] Step 3 specifically includes the following sub-steps:

[0056] (3.1) See Figure 2 To create a car styling evaluation dataset corresponding to the multi-view car image dataset, collect and organize valid user comment samples (including user styling ratings, user styling semantic evaluations, user professionalism, and user objectivity) corresponding to the multi-view car image dataset from step 1. Then, process the user ratings using a reasonable comprehensive scoring mechanism: first, use the Grubbs test to remove outliers from the user styling ratings. The formula is as follows:

[0057]

[0058] Among them G i This represents the Grubbs test statistic, where s is the sample standard deviation. The average value is the sampled average, where i represents any data point, and x represents the average value. iThis represents the score for this data point, and each score value x is calculated according to the formula. i G i The value is compared with the Grubbs critical table determined based on the sample size and confidence level. If the obtained G... i If the value is greater than the value in the table, then x is considered to be... i Remove outliers;

[0059] Secondly, user attributes are quantified. Decision-makers recognize that users possess multiple dimensional attributes, and the measurements of these attributes are transformed into weighting factors. The weighting principle is: the higher the user's dimensional attribute value, the higher the weighting factor. Since the credibility weight and the user's professionalism follow a normal distribution, and it is assumed that the weighting factors can be sampled from the probability distribution density function, a one-dimensional weighted Gaussian distribution function is defined:

[0060]

[0061] Where W is the initial weighting factor, x is the quantified value of user attributes, σ is the standard deviation, μ is the quantified factor of the highest user professionalism, A is the amplitude elasticity coefficient, and B controls the flatness of the function. Considering multiple independent user attribute dimensions, the one-dimensional weighted Gaussian distribution function can be extended to the multi-dimensional case, using the following formula:

[0062]

[0063] Where W is the weighting factor for calculating the rating of a certain car model, i represents the user attribute dimension, n represents the number of dimensions, and A i B is the amplitude elasticity coefficient. i To adjust the difference between the maximum and minimum values ​​of the car styling score weights, x i σ represents the quantified values ​​of each user attribute. i μ is the standard deviation of the quantification levels of each user attribute. i The highest quantified level for each user attribute. For a specific car model, assuming the number of users evaluating it is n, the weight of any user after attribute quantification and normalization is:

[0064]

[0065] Where i represents a single user, W i The weighting factor is calculated based on attributes for a single user. If the weighted rating is assigned to a single user, then the weighted score for a particular car model is the weighted sum of the single user's rating and its corresponding weight. The processed car styling user ratings are used to annotate the multi-view car image dataset with user ratings, resulting in a car styling user rating dataset. The dataset is then split into a training set and a validation set according to a ratio K (e.g., K = 7 / 3).

[0066] (3.2) Collect and organize user semantic evaluation data corresponding to the large-scale multi-view car image dataset in step 1. Extract and analyze car styling semantic evaluation keywords from the user semantic evaluation data. For example, use the TF-IDF algorithm to obtain keywords for styling semantic evaluation, as shown in the formula:

[0067]

[0068] Where i represents a specific word, TF i This indicates the word frequency, n i This indicates the number of times the word appears in the text, where m is the number of words in the comment text, and k is the number of words in the text. i This indicates the number of times the word or phrase appears. Then, the formula is used:

[0069]

[0070] Calculate the inverse document frequency (IDF) of the word. i Where d represents the number of comments containing the keyword, and D represents the total number of comments. Let TF-IDF = TF i ×IDF i The value measures the importance of terms in styling reviews, enabling the extraction of semantic evaluation keywords for styling. The extracted semantic evaluation keywords are then used to semantically annotate the multi-view car image dataset from step 1, resulting in a car exterior styling user semantic evaluation dataset. This dataset is then split into training and validation sets according to a ratio K (e.g., K = 7 / 3).

[0071] (3.3) Collect and organize valid expert comment samples corresponding to the multi-view car image dataset in Step 1 (including expert styling scores, expert styling semantic evaluations, expert years of experience, number of car models commented on by experts, etc.), and then use the Grubbs test to remove outliers from the expert styling scores. The formula is as follows:

[0072]

[0073] Where s is the sample standard deviation. The average value is calculated for each sample value x according to the formula. i G i The value is compared with the Grubbs critical table determined based on the sample size and confidence level. If the obtained G... i If the value is greater than the value in the table, then x is considered to be... i Remove outliers;

[0074] Secondly, the expert attributes are quantified. When an expert has more than N1 years of experience in automotive exterior design or evaluation (e.g., N1 > 5 years) and has evaluated more than N2 models (e.g., N2 > 100 models), the expert score can be considered objective and reasonable. The processed automotive styling expert scores are then used to annotate the multi-view automotive image dataset with expert scores, resulting in an automotive styling expert score dataset. The dataset is then split into a training set and a validation set according to a ratio K (e.g., K = 7 / 3).

[0075] (3.4) Collect and organize expert semantic evaluation data corresponding to the large-scale multi-view car image dataset in step 1. Extract and analyze car styling semantic evaluation keywords from the expert semantic evaluation data. For example, use the TF-IDF algorithm to obtain keywords for styling semantic evaluation, as shown in the formula:

[0076]

[0077] Where i represents a specific word, TF i This indicates the word frequency, n i This indicates the number of times the word appears in the text, where m is the number of words in the comment text, and k is the number of words in the text. i This indicates the number of times the word or phrase appears. Then, the formula is used:

[0078]

[0079] Calculate the inverse document frequency (IDF) of the word. i Where d represents the number of comments containing the keyword, and D represents the total number of comments. Let TF-IDF = TF i ×IDF i The value measures the importance of terms in styling reviews, enabling the extraction of styling semantic evaluation keywords. The extracted styling semantic evaluation keywords are then used to perform corresponding semantic annotation on a multi-view automotive image dataset, resulting in an automotive styling expert semantic evaluation dataset. This dataset is then split into training and validation sets according to a ratio K (e.g., K = 7 / 3).

[0080] Step 4 specifically includes the following sub-steps:

[0081] (4.1) See reference Figure 3The user rating dataset for car exterior styling in step (3.1) is classified according to the car class classification method in step (1.2) to obtain a user rating subset for car exterior styling of class X, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV. A mapping relationship between styling scores and car exterior styling features in the subset is established using deep learning regression. Deep learning network models are trained separately to obtain an automatic user rating machine for car exterior styling of class X corresponding to the subset. The user styling score dataset is split into training and testing sets according to a ratio K (K = 7 / 3). The car styling score labels are normalized to between 0 and 1. The last layer of the network is connected to the Sigmoid function for logistic regression calculation and fitted to the styling score points on the Sigmoid function curve so that the trained model can predict and output car styling scores. The parameters for training a deep learning model are as follows: epoch n1, training batch n2, initial learning rate n3, the learning rate is adjusted accordingly as the epoch ranges, momentum n4, and weight decay n5; (e.g., n1 = 100, n2 = 32, n3 = 0.0001, n4 = 0.9, n5 = 0.0005). See also... Figure 4 This process involves calculating adjustment parameters for car ratings at different levels. For a given car styling dataset with multi-view images at a specific level, rating predictions are performed, and the accuracy of the predicted ratings is measured using MAE (Mean Absolute Error). This yields statistical data on the ratings of various car models at different angles within the same car level, which is then compiled into a car styling rating statistics table. The angles with the lowest MAE for each car model are selected and represented as a set. Calculate the scoring adjustment parameters for each angle, i.e., the frequency of occurrence of each angle:

[0082]

[0083] Where θ represents the angle, and i represents the specific angle. This indicates the error at that angle. Let represent the scoring adjustment parameters, and m represent the number of measured angle samples. The angle with the highest frequency indicates a higher reliability of the score at that point. Therefore, the set of car styling scoring adjustment parameters for this level is: The adjustment parameters for car ratings differ across different levels. Therefore, predictions and calculations should be performed separately for images in each sub-database. When rating car styling, the corresponding angle adjustment parameters are selected to adjust the score, resulting in a weighted score for car styling angles. The formula is as follows:

[0084]

[0085] Where X represents the final car styling score. This indicates that the weighting of the scoring from the corresponding angle will be adjusted. This indicates the predicted score from that angle;

[0086] (4.2) See reference Figure 3 The car exterior styling expert rating dataset from step (3.3) is classified according to the car class classification method in step (1.2) to obtain the X-level car exterior styling expert rating subset, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV. A deep learning regression method is used to establish the mapping relationship between the styling rating and the car exterior styling features in the subset. Deep learning network models are trained separately to obtain the X-level car exterior styling user automatic rating machine corresponding to the subset. The user styling rating dataset is split into training set and test set according to the ratio K (e.g., K = 7 / 3). The car styling rating labels are normalized to between 0 and 1. The last layer of the network is connected to the Sigmoid function to perform logistic regression calculation and fit to the styling rating points on the Sigmoid function curve so that the trained model can predict the output car styling rating. The parameters for training a deep learning model are as follows: epoch n1, training batch n2, initial learning rate n3, the learning rate is adjusted accordingly as the epoch ranges, momentum n4, and weight decay n5; (e.g., n1 = 100, n2 = 32, n3 = 0.0001, n4 = 0.9, n5 = 0.0005). See also... Figure 4 This process involves calculating adjustment parameters for car ratings at different levels. For a given car styling dataset with multi-view images at a specific level, rating predictions are performed, and the accuracy of the predicted ratings is measured using MAE (Mean Absolute Error). This yields statistical data on the ratings of various car models at different angles within the same car level, which is then compiled into a car styling rating statistics table. The angles with the lowest MAE for each car model are selected and represented as a set. Calculate the scoring adjustment parameters for each angle, i.e., the frequency of occurrence of each angle:

[0087]

[0088] Where θ represents the angle, and i represents the specific angle. This indicates the error at that angle. Let represent the scoring adjustment parameters, and m represent the number of measured angle samples. The angle with the highest frequency indicates a higher reliability of the score at that point. Therefore, the set of car styling scoring adjustment parameters for this level is: The adjustment parameters for car ratings differ across different levels. Therefore, predictions and calculations should be performed separately for images in each sub-database. When rating car styling, the corresponding angle adjustment parameters are selected to adjust the score, resulting in a weighted score for car styling angles. The formula is as follows:

[0089]

[0090] Where X represents the final car styling score. This indicates that the weighting of the scoring from the corresponding angle will be adjusted. This indicates the predicted score from that angle;

[0091] Step 5 specifically includes the following sub-steps:

[0092] (5.1) See Figure 5 The user semantic evaluation dataset of car exterior styling in step (3.2) is classified according to the car level classification method in step (1.2) to obtain the user rating subset of X-level car exterior styling, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV. A deep learning multi-label classification method is used to establish the mapping relationship between the styling semantic evaluation and the car exterior styling features in the subset. Deep learning multi-label classification models are trained to obtain the automatic semantic evaluation machine for X-level car exterior styling corresponding to the subset. Since the exterior styling of a car model is not limited to a single semantic feature, a multi-label classification method is used to output multiple semantic evaluations of the car exterior styling. The neurons in the output layer of the deep learning network are set according to the corresponding number of semantic evaluation labels. Because each classification label in multi-label classification is independent but not mutually exclusive, i.e., there is a certain correlation between the labels, the Sigmoid activation function is used to handle this type of problem, converting the result of each classification calculation into a probability value between 0 and 1. Since the long-tail effect of the dataset can negatively affect model training, Focal Loss (FL) loss function is used to improve it. At the same time, the ratio of positive and negative samples is controlled by the inverse class frequency, which solves the problem of imbalance between positive and negative samples and the problem of distinguishing between simple and complex samples, and improves the model's classification effect on long-tail labels. The user modeling semantic evaluation dataset is split into training set and test set according to the ratio K (e.g., K = 7 / 3).

[0093] (5.2) See Figure 5The semantic evaluation dataset of car exterior styling experts from step (3.4) is classified according to the car classification method in step (1.2) to obtain the user rating subset of X-level car exterior styling, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV. A mapping relationship between styling semantic evaluation and car exterior styling features in the subset is established using a deep learning multi-label classification method. Deep learning multi-label classification models are trained to obtain the automatic semantic evaluation machine of X-level car exterior styling experts corresponding to the subset. Since the exterior styling of a car model is not limited to a single semantic feature, a multi-label classification method is used to output multiple semantic evaluations of the car exterior styling. The neurons in the output layer of the deep learning network are set according to the corresponding number of semantic evaluation labels. Because each classification label in multi-label classification is independent but not mutually exclusive, i.e., there is a certain correlation between labels, the Sigmoid activation function is used to handle this type of problem, converting the result of each classification calculation into a probability value between 0 and 1. Since the long-tail effect of the dataset can negatively impact model training, Focal Loss (FL) is used to improve it. Simultaneously, inverse class frequency is used to control the ratio of positive to negative samples, resolving the imbalance between positive and negative samples and the problem of distinguishing between simple and complex samples, thus improving the model's classification performance on long-tail labels. The expert modeling semantic evaluation dataset is split into training and test sets according to a ratio K (e.g., K is 7 / 3). Step 6 specifically includes the following sub-steps:

[0094] (6.1) See reference Figure 6 This paper employs effective deep learning model visualization methods, such as Gradient Class Weighted Activation Mapping (Grad-CAM), to visualize the salient features of car styling. This method uses gradient information flowing into the convolutional layers of a CNN to understand the importance of each neuron in determining the prediction target. Taking a classification model as an example, for the final output c (where c represents the class or regression value), the weight formula for the Kth neuron in the last convolutional layer is as follows:

[0095]

[0096] in, is the sensitivity of c relative to the k-th channel of the output feature map of the last convolutional layer, Z is the number of pixels in the feature map, i and j represent the indices of the width and height dimensions, respectively, and y c This represents the probability of c in the output of the last activation function. This is the feature map output by the last convolutional layer, where k is the index of the channel dimension of the feature map. This involves taking the partial derivative of that probability value with respect to all pixels in the output feature map of the k-th neuron in the last layer. Then... The feature maps output from the last convolutional layer are weighted and linearly combined, and then processed by the ReLU activation function to filter out negative values ​​obtained by weighting at a certain position on feature map A, retaining only the outputs positively correlated with the predicted value. The overall formula is as follows:

[0097]

[0098] Where L is a two-dimensional category activation heatmap, A k It is the feature map output by the last convolutional layer. By superimposing the heatmap on the original image, a visualization effect of significant features is presented.

[0099] (6.2) Using the deep learning model visualization method introduced in step (6.1), visualize the salient features of the car exterior styling user automatic rating machine, user automatic semantic evaluation machine, expert automatic rating machine, and expert automatic semantic evaluation machine obtained in steps (4.1), (4.2), (5.1), and (5.5) respectively, and locate the car exterior styling features that users and experts are interested in in the form of heat maps.

[0100] Step 7 specifically includes the following sub-steps:

[0101] (7.1) For the input car exterior design rendering, the car exterior angle recognition machine and car exterior level recognition machine obtained in step (2.1) are used to determine the car model angle and car model level;

[0102] (7.2) After determining the angle and level of the vehicle model in step (7.1), the user score is obtained by the user automatic scoring machine corresponding in step (4.1), and the expert score is obtained by the expert automatic scoring machine corresponding in step (5.1). The saliency feature visualization method in step (6.1) is used to output the vehicle exterior styling features that the user automatic scoring machine and the expert automatic scoring machine focus on when scoring.

[0103] (7.3) After determining the angle and level of the vehicle model in step (7.1), the user semantic evaluation is obtained by the user automatic semantic evaluation machine corresponding in step (4.2), and the expert semantic evaluation is obtained by the expert automatic semantic evaluation machine corresponding in step (5.2). The saliency feature visualization method in step (6.1) is used to output the vehicle exterior styling features that the user automatic semantic evaluation machine and the expert automatic semantic evaluation machine pay attention to when performing semantic evaluation of the vehicle exterior styling.

Claims

1. A big data-driven automatic evaluation method for automotive exterior styling design, characterized in that, The steps are as follows: Step 1: Create a large-scale multi-view car image dataset for car angle recognition and grade recognition, and annotate relevant information; (1.1) Collect and organize multi-angle images of car exteriors from different brands and models, with a total sample size of no less than N1 images, covering no fewer than [number missing] car brands. N Two models, with a model year spanning [number missing]. Y 1~ Y Over two years, samples were taken from each car model at a horizontal angle, rotating around the perimeter at a uniform speed. N 3 pictures; (1.2) Label each of the multi-view car exterior images in step (1.1), including car class, brand, model and car angle. The car class is divided into four levels: sedans, A, B, C and D, and SUVs, compact, mid-size, mid-size and large SUVs. This results in a large-scale multi-view car image dataset, which is then split into corresponding sub-datasets according to car class. Step 2: Train the multi-view car image dataset from Step 1 for angle recognition and grade recognition respectively, and obtain the car angle recognition machine and car grade recognition machine accordingly. (2.1) The subsets of data split in step 1 are processed proportionally. K The system is divided into corresponding training and validation sets. A deep learning regression network is used to train the car angle recognition machine, and a deep learning classification network is used to train the car grade recognition machine. (2.2) Use the car angle recognition machine and car grade recognition machine obtained in step (2.1) to quickly and accurately identify the input car exterior image and predict the angle and grade of the input car image; Step 3: Create a car exterior styling scoring dataset containing user and expert levels for training the automatic car exterior styling scoring machine, and a car exterior styling semantic evaluation dataset containing user and expert levels for training the automatic car exterior styling semantic evaluation machine. Process the multi-view car image dataset from Step 1, and divide it into two groups: end users and styling experts. Perform styling scoring annotation and styling semantic evaluation annotation on the multi-view car image dataset respectively, to obtain the user car styling evaluation dataset containing the car exterior styling user scoring dataset and the car exterior styling user semantic evaluation dataset, and the expert car styling evaluation dataset containing the car exterior styling expert scoring dataset and the car exterior styling expert semantic evaluation dataset. (3.1) Collect and organize user rating data corresponding to the multi-view car image dataset in step 1, clean the user rating data, analyze and quantify the multi-dimensional attributes of users, assign different rating weights according to different user attributes, and obtain the comprehensive user rating of car exterior styling. Using this comprehensive user rating, user ratings are assigned to car images in the multi-view car image dataset to obtain a car exterior design user rating dataset. (3.2) Collect and organize user semantic evaluation data corresponding to the multi-view car image dataset in step 1, extract and analyze car styling semantic evaluation keywords from the user semantic evaluation data, and then perform one-to-one user semantic evaluation annotation on the car images in the multi-view car image dataset to obtain the car exterior styling user semantic evaluation dataset. (3.3) Collect and organize the expert rating data corresponding to the multi-view car image dataset in step 1, analyze the reliability of the expert evaluation based on the experts' years of experience and the number of car models evaluated, and clean the expert rating data. Using this expert rating data, one-to-one expert rating annotations were performed on the car images in the multi-view car image dataset to obtain the car exterior styling expert rating dataset. (3.4) Collect and organize expert semantic evaluation data corresponding to the multi-view car image dataset in step 1, extract and analyze car styling semantic evaluation keywords from the expert semantic evaluation data, and then perform one-to-one expert semantic evaluation annotation on the car images in the multi-view car image dataset to obtain the car exterior styling expert semantic evaluation dataset. Step 4: Split the car exterior styling rating dataset created in Step 3 into corresponding sub-datasets according to car grade and evaluator attributes. Use deep learning regression methods to train the user and expert levels respectively to obtain the car exterior styling user automatic rating machine and the car exterior styling expert automatic rating machine. (4.1) Classify the user rating dataset of car exterior styling in step (3.1) according to the car level classification method in step (1.2) to obtain the user rating subset of car exterior styling of X-level models, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning regression method to establish the mapping relationship between styling scores and car exterior styling features in the subset, train deep learning network models respectively, and obtain the automatic user rating machine of car exterior styling of X-level models corresponding to the subset; (4.2) Classify the car exterior styling expert scoring dataset in step (3.3) according to the car level classification method in step (1.2) to obtain the X-level car exterior styling expert scoring subset, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning regression method to establish the mapping relationship between styling scores and car exterior styling features in the subset, train deep learning network models respectively, and obtain the X-level car exterior styling expert automatic scoring machine corresponding to the subset; Step 5: Split the car exterior styling semantic evaluation dataset created in Step 3 into corresponding sub-datasets according to car grade and evaluator attributes. Use deep learning multi-label classification method to train the user and expert levels respectively to obtain the car exterior styling user automatic semantic evaluation machine and the car exterior styling expert automatic semantic evaluation machine. (5.1) Classify the user semantic evaluation dataset of car exterior styling in step (3.2) according to the car level classification method in step (1.2) to obtain the user semantic evaluation subset of X-level car exterior styling, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning multi-label classification method to establish the mapping relationship between styling semantic evaluation and car exterior styling features in the subset, train deep learning multi-label classification models respectively, and obtain the automatic semantic evaluation machine of user exterior styling of X-level car exterior styling corresponding to the subset; (5.2) Classify the automotive exterior styling expert semantic evaluation dataset in step (3.4) according to the automotive class classification method in step (1.2) to obtain the X-level vehicle exterior styling expert semantic evaluation sub-dataset, where X is A, B, C, D, compact SUV, mid-size SUV, mid-to-large SUV, and large SUV; use deep learning multi-label classification method to establish the mapping relationship between styling semantic evaluation and automotive exterior styling features in the sub-dataset, train deep learning multi-label classification models respectively, and obtain the X-level vehicle exterior styling expert automatic semantic evaluation machine corresponding to the sub-dataset; Step 6: Use deep learning model visualization methods to visualize the salient features of the automatic car exterior styling scoring machine and the automatic car exterior styling semantic evaluation machine, respectively. (6.1) A deep learning model visualization method is used to visualize the salient features of car styling; this method uses gradient information flowing into the convolutional layers of the CNN to understand the importance of each neuron in the prediction target decision, and visualizes the final output results. c , c Represents the category or regression value, the th in the last convolutional layer K The weight formula for each neuron is as follows: in, yes c The first feature map relative to the output feature map of the last convolutional layer k The sensitivity of each channel, Z It is the number of pixels in the feature map. i and j These represent the indexes of the width and height dimensions, respectively. y c This indicates the output of the last activation function. c The probability, It is the feature map output by the last convolutional layer. k It is the index of the channel dimension of the feature map. This probability value applies to the last layer. K Calculate the partial derivatives of all pixels in the feature map output by each neuron; then... The feature maps output from the last convolutional layer are weighted and linearly combined, and then processed by the ReLU activation function to filter out the feature maps. A The negative values ​​obtained by weighting at a certain position are used to retain only the outputs that are positively correlated with the predicted values. The overall formula is as follows: in, It is a two-dimensional category activation heatmap. It is the feature map output by the last convolutional layer. By superimposing the heatmap on the original image, a visualization effect of significant features is presented. (6.2) Using the deep learning model visualization method introduced in step (6.1), visualize the salient features of the car exterior styling user automatic rating machine, user automatic semantic evaluation machine, expert automatic rating machine, and expert automatic semantic evaluation machine obtained in steps (4.1), (4.2), (5.1), and (5.5) respectively, and locate the car exterior styling features that users and experts are interested in in the form of heat maps; Step 7: For the input car exterior design renderings, use the automatic car exterior scoring machine and automatic semantic evaluation machine obtained in Steps 2 and 4-6 to provide designers with real-time objective exterior scoring, semantic evaluation and corresponding salient feature visualization information from both user and styling expert perspectives. (7.1) For the input car exterior design rendering, the car exterior angle recognition machine and car exterior level recognition machine obtained in step (2.1) are used to determine the car model angle and car model level; (7.2) After determining the angle and level of the vehicle model in step (7.1), the user score is obtained by the user automatic scoring machine corresponding in step (4.1), and the expert score is obtained by the expert automatic scoring machine corresponding in step (5.1). The saliency feature visualization method in step (6.1) is used to output the vehicle exterior design features that the user automatic scoring machine and the expert automatic scoring machine focus on when scoring. (7.3) After determining the angle and level of the vehicle model in step (7.1), the user semantic evaluation is obtained by the user automatic semantic evaluation machine corresponding in step (4.2), and the expert semantic evaluation is obtained by the expert automatic semantic evaluation machine corresponding in step (5.2). The saliency feature visualization method in step (6.1) is used to output the vehicle exterior styling features that the user automatic semantic evaluation machine and the expert automatic semantic evaluation machine pay attention to when performing semantic evaluation of the vehicle exterior styling.

Images