A used car evaluation and prediction method based on a multi-modal algorithm model

Abstract

The application provides a used car evaluation and prediction method based on a multi-modal algorithm model, which comprises the following steps: S1, multi-modal data collection and preprocessing; S2, processing the data processed by the step S1 by using a multi-modal algorithm model; and S3, modal fusion processing based on the step S2. The application can not only effectively process complex multi-modal data, but also improve the operation efficiency of the model while maintaining high accuracy. The application performs well in processing complex data, greatly improves the evaluation and prediction performance by integrating image, text and numerical data, and realizes comprehensive evaluation and accurate prediction of used cars. The application can also provide personalized price prediction and quality analysis to help users make more intelligent purchase decisions.

Description

Technical Field

[0001] This invention belongs to the field of vehicle detection and analysis technology, and specifically relates to a method for used car evaluation and prediction based on a multimodal algorithm model. Background Technology

[0002] With the rapid development of the used car market, consumer demand for used cars is constantly increasing, and the market size is expanding year by year. However, due to the complex origins and varying conditions of used cars, accurately assessing their value and predicting market demand has become extremely important. Traditional used car assessment methods mainly rely on human experience and simple statistical models, which are insufficient to accurately reflect vehicle condition and market dynamics, and cannot meet the needs of the modern market.

[0003] Currently, data science and artificial intelligence technologies are being applied more and more widely in various fields, especially achieving remarkable results in big data processing and analysis. In the used car sector, big data and artificial intelligence technologies can be used to achieve more accurate assessments and predictions. While existing solutions, such as used car assessment methods based on machine learning algorithms like linear regression and decision trees, have improved accuracy to some extent, they still have many shortcomings:

[0004] Insufficient data utilization: Existing methods mainly rely on single-modal data (such as numerical data) and ignore other important information such as images and text, resulting in inaccurate evaluation results.

[0005] Limited model performance: Traditional machine learning models such as linear regression and decision trees have limited performance when dealing with complex used car data and cannot fully capture the deep features in the data.

[0006] High computational resource requirements: Some complex models (such as deep learning models) require a lot of computational resources during training and prediction, resulting in high costs and making them unsuitable for widespread application in practice.

[0007] Significant human intervention is required: Many existing methods still require a large amount of human intervention, especially in the data processing and feature extraction stages, which is inefficient and prone to introducing human error. Summary of the Invention

[0008] To address the aforementioned problems, this invention provides a method for used car evaluation and prediction based on a multimodal algorithm model.

[0009] This invention provides a method for used car evaluation and prediction based on a multimodal algorithm model, comprising the following steps:

[0010] S1. Multimodal data collection and preprocessing;

[0011] S2. Process the data processed in step S1 using a multimodal algorithm model;

[0012] S3. Perform modal fusion processing based on step S2.

[0013] further,

[0014] Step S1 includes:

[0015] For text data, remove punctuation marks and standardize capitalization to clean and standardize the text content;

[0016] For numerical data, fill in the missing values ​​with the mean, median, or mode of the values ​​other than the missing values;

[0017] For image data, i.e., pixel values, we remove noise, adjust brightness and size, and improve image quality. Here, we use filters to smooth the image and remove noise.

[0018] For the text data, numerical data, and image data, perform data deduplication: delete duplicate samples in the dataset to ensure that each data point has no duplicate values.

[0019] further,

[0020] The filter is a Gaussian filter or a median filter.

[0021] further,

[0022] Step S1 further includes the following step:

[0023] S1a. Outliers in the image data are processed as follows: Outliers are detected using the quartile range method. A quartile is a statistical measure that divides a dataset into four equal parts. Specifically, the quartile range method includes:

[0024] Let Q1 be the first quartile, representing the 25th percentile of the dataset consisting of the image data, that is, 25% of the data points in the dataset are below this value;

[0025] Let Q3 be the third quartile, representing the 75th percentile of the dataset consisting of the image data, that is, 75% of the data points in the dataset are below this value;

[0026] Let IQR = Q3 - Q1, where IQR is the difference between Q3 and Q1, representing the middle 50% distribution range of the dataset. Then, we have a lower bound Lm = Q1 - 1.5 × IQR and an upper bound Hm = Q3 + 1.5 × IQR. When the value of a data point in the dataset is lower than the lower bound Lm or higher than the upper bound Hm, the value of the data point is an outlier, and the outlier is deleted.

[0027] further,

[0028] In step S2,

[0029] The image data is preprocessed using a convolutional neural network, including the following steps:

[0030] S211. Image preprocessing, including the following steps:

[0031] Adjust image size: Resize all images to the same size to ensure consistent input format. Specifically, scale the pixel values ​​of the images from [0, 225] to [0, 1], i.e., normalize the pixels. The formula is: Np = pv / 255.

[0032] In the above formula, pv is the pixel value in the image, and Np is the normalized pixel value;

[0033] Data augmentation: Randomly rotate, crop, and flip the image.

[0034] S212. Construct a convolutional neural network model, including the following steps:

[0035] The convolution operation is set up using the following formula:

[0036]

[0037] Let the specific pixel coordinates of the input image be the initial pixel position, and the specific pixel coordinates of the convolution kernel be the convolution kernel pixel position. Then, in the above formula, F(i,j) is the output value at the initial pixel position (i,j) after the convolution operation; I(i+m,j+n) is the pixel value of the input image at the initial pixel position (i+m,j+n); K(m,n) is the weight of the convolution kernel at the position (m,n); M and N are the height and width of the convolution kernel; the scale factor α is used to adjust the output amplitude of the convolution operation, and the bias term β is used to adjust the baseline value of the output of the convolution operation, where i,j,m,n are integers, and M and N are positive integers.

[0038] Set the activation function:

[0039]

[0040] In the above formula, α1 is a smoothing parameter, which depends on the specific working conditions.

[0041] A pooling operation is configured, using a fixed-size pooling window in the feature pool. The maximum value is selected within each window, i.e., max pooling. To maintain the feature map size matching the input image size, extra pixels are added to the boundaries of the input image and padded with zeros.

[0042] In classification or regression, this step first flattens the multidimensional feature map after convolution and pooling into a one-dimensional vector. This flattening operation transforms two-dimensional data into one-dimensional data, which is then input into the fully connected layer. The fully connected layer generates new data by multiplying each input data point by a weight and adding a bias value, thus transforming the feature map data into the final result format.

[0043] The model's output varies depending on the simulation task. If the simulation is for classification, the softmax function is used to convert the results of the fully connected layers into probabilities for each class, and the class with the highest probability is output. If the simulation is for regression, a continuous numerical result is directly output to represent the predicted value.

[0044] The convolutional neural network model constructed in step S212 processes the data obtained in step S211. A fixed-length feature vector, denoted as the first feature vector, is extracted from the last layer (pooling layer) of the convolutional neural network model for each image. When there are n1 images in the multimodal data, n1 is an integer greater than 1. After processing the multimodal data in step S211, the data is input into the convolutional neural network model to obtain n1 first feature vectors. The n1 first feature vectors are assigned different weights and then summed to obtain a total feature vector, denoted as the image feature vector. The weights are set based on the importance of image features in the final task.

[0045] further,

[0046] In step S2,

[0047] The text data is preprocessed using a recurrent neural network, including the following steps:

[0048] S221. Text preprocessing, including the following steps:

[0049] Text cleaning: Remove punctuation marks, numbers, and special characters from the text;

[0050] Word segmentation: The text is divided into words or phrases to obtain a text sequence;

[0051] Create a vocabulary: Create a vocabulary for used car evaluation and prediction, and assign a unique index to each word in it;

[0052] Sequence encoding: Based on the index, the text sequence is correspondingly converted into an index sequence;

[0053] All the index sequences are padded or truncated to ensure they have the same length.

[0054] S222. Construct a recurrent neural network model, including the following steps:

[0055] The embedding layer is configured such that its input is an index sequence, where each index represents a word; the embedding layer performs the following operations: searching the embedding matrix and converting the indexes into corresponding embedding vectors; the embedding layer outputs an embedding vector sequence, where each embedding vector represents a high-dimensional semantic representation of the word.

[0056] Set up a recurrent neural network layer: The input of the recurrent neural network layer is the embedding vector sequence; The operation performed by the recurrent neural network layer is: At each time step, the recurrent neural network layer accepts the current input and the hidden state of the previous time step, and calculates a new hidden state; The output of the recurrent neural network layer is a hidden state sequence, wherein each hidden state contains information from the beginning of the hidden state sequence to the current time step;

[0057] Set the activation function: Same as equation (1),

[0058] S223. The data obtained in step S221 is processed by the recurrent neural network model. A segment of continuous text data in the multimodal data is given a feature vector of fixed length, denoted as the second feature vector. When there are n2 segments of continuous text data in the multimodal data, n2 is an integer greater than 1. The multimodal data is processed by step S221 and then input into the recurrent neural network model to obtain n2 second feature vectors. The n2 second feature vectors are added together to obtain a total feature vector, denoted as the text feature vector.

[0059] further,

[0060] In step S2,

[0061] The numerical data is preprocessed using a linear regression model, including the following steps:

[0062] S231. Numerical data preprocessing, including the following steps:

[0063] Identify missing values ​​in the numerical data;

[0064] Fill in the missing data;

[0065] Identify and process outliers in the numerical data.

[0066] S232. Construct a linear regression model, including creating a linear regression model instance using the LinearRegression class.

[0067] S233. The linear regression model constructed in step S232 is used to process the data obtained in step S231. In the multimodal data, a fixed-length feature vector is obtained for each numerical data point, denoted as the third feature vector. When there are n3 numerical data points in the multimodal data, n3 is an integer greater than 1. The multimodal data is processed by step S231 and then input into the linear regression model to obtain n3 third feature vectors. The n3 third feature vectors are added together to obtain a total feature vector, denoted as the numerical feature vector.

[0068] further,

[0069] In step S231, the quartile range method is used to process and calculate the upper and lower bounds of outliers in the numerical data. That is, after replacing the image data in step S1a with the numerical data, step S1a is executed again, and then the numerical data that exceeds the upper and lower bounds is removed.

[0070] further,

[0071] Step S3 includes the following steps:

[0072] Data integration: The image feature vector, text feature vector, and numerical feature vector are concatenated into a comprehensive feature vector;

[0073] Setting up a fully connected neural network model involves the following steps:

[0074] Set an input layer whose size is the same as the length of the synthesized feature vector;

[0075] A hidden layer is set up, and at least one hidden layer is added after the input layer. After receiving the input data, the neurons in each hidden layer linearly transform the data by weights and biases.

[0076] An output layer is set up. In used car-related tasks, the output results of the output layer depend on the specific business requirements and are divided into the following three categories:

[0077] Predicting Used Car Prices: If the task is to predict the selling price of a used car, the output layer has one neuron that uses a linear activation function to output a continuous value, which represents the predicted used car price.

[0078] Predicting the type of used car: If the task is to classify the type of used car, the output layer has multiple neurons, and each type corresponds one-to-one with one of the multiple neurons. An activation function is used to output a probability distribution, where the probability represents the probability that the input used car belongs to a different type.

[0079] Used car quality classification: If the task is to evaluate the quality of a used car, and the quality is divided into three levels: good, medium, and poor, then the output layer has three neurons, which use activation functions to output three probability values, representing the probability that the used car belongs to each quality level.

[0080] This invention provides a used car evaluation and prediction method based on multimodal algorithm models. It designs a series of efficient multimodal algorithm models, integrating image, text, and numerical data to achieve comprehensive evaluation and accurate prediction of used cars. These models not only effectively handle complex multimodal data but also improve operational efficiency while maintaining high accuracy. This invention performs exceptionally well when processing complex data, significantly enhancing evaluation and prediction performance. Furthermore, this invention not only improves the accuracy of used car market value assessment but also provides personalized price predictions and quality analysis, helping users make more informed purchasing decisions. In addition, this invention reduces manual intervention, improving work efficiency and consistency.

[0081] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0082] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0083] Figure 1 A flowchart of a used car evaluation and prediction method based on a multimodal algorithm model according to an embodiment of the present invention is shown. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. The terms "first," "second," "third," etc., in the specification, claims, or accompanying drawings of this application are used to distinguish different objects, not to describe a specific order or hierarchy. The term "multiple" in this application refers to two or more (including two).

[0086] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0087] Figure 1 This invention provides a flowchart of a used car evaluation and prediction method based on a multimodal algorithm model. The method includes the following steps, wherein each model mentioned below needs to be trained, validated, and tested to complete the evaluation and optimization of each model. The training, validation, and testing methods, as well as the further evaluation and optimization methods, are known to those skilled in the art and are not the focus of this application; therefore, they will not be elaborated upon here.

[0088] S1. Multimodal data collection and preprocessing.

[0089] Multimodal Data Collection: This involves collecting used car-related data (multimodal data) from multiple channels, including vehicle history data, market transaction data, vehicle images, and user reviews. This data falls into different modalities such as text, images, and audio. These channels include online used car trading platforms, car auction houses, dealerships, market research, third-party data providers, social media, and discussions and user reviews on online car enthusiast forums. The relevant data includes numerical data such as vehicle price and mileage, image data such as vehicle exterior photos, and text data such as user reviews and vehicle condition descriptions. Specifically, the multimodal used car-related data should include at least tens of thousands of records of numerical data to capture changes in vehicle price, year, mileage, and other characteristics; at least several thousand high-quality vehicle photos covering different brands, models, and conditions; and tens of thousands of comments to encompass different user perspectives and descriptive styles. Furthermore, multimodal data for model implementation needs to be prepared. This data is divided into training, validation, and test sets for training, validating, and testing the various models to evaluate and optimize them.

[0090] Multimodal data preprocessing: This involves cleaning, standardizing, and enhancing different modalities of multimodal data, such as text, images, and audio, to ensure data quality and consistency. Cleaning includes the following steps:

[0091] S11. For text data, remove punctuation and unify case to clean and standardize the text content. For example, the original text is "Hello world! This is a test, data cleaning. We need to clean the data: remove punctuation; unify case.", and the cleaned text is "Hello world, this is a test, data cleaning. We need to clean the data, remove punctuation, and unify case."

[0092] S12. For numerical data, fill in the missing values ​​with the mean, median, or mode of the remaining values. For example, Table 1 below provides three variables: variable A, variable B, and variable C. Each variable...

[0093] Table 1

[0094] VariableA Variable B Variable C 10 20 NaN 15 NaN 25 NaN 30 35 40 45 50

[0095] Four values ​​can be selected, but one value is missing, denoted as NaN. If the missing value is filled by the mean of the values ​​excluding the missing value, then the mean of the values ​​excluding the missing value for variable A is (10+15+40) / 3 = 65 / 3 ≈ 21.67. Similarly, the mean values ​​excluding the missing values ​​for variables B and C are 31.67 and 36.67, respectively. Table 2 is obtained after filling the missing values ​​in Table 1.

[0096] Table 2

[0097] VariableA Variable B Variable C 10 20 36.67 15 31.67 25 21.67 30 35 40 45 50

[0098] S13. For image data, i.e., pixel values, noise is removed, brightness and size are adjusted, and image quality is improved. Filters such as Gaussian filters and median filters are used to smooth the image and remove noise. These filters are tools used in signal processing and image processing to remove noise or highlight certain features. Outliers in the image data are processed as follows: the Interquartile Range (IQR) method is used to detect outliers. The quartiles are statistics that divide a dataset into four equal parts. Specifically, the IQR method includes:

[0099] Let Q1 be the first quartile, representing the 25th percentile of the dataset consisting of image data, that is, 25% of the data points in the dataset are below this value;

[0100] Let Q3 be the third quartile, representing the 75th percentile of the dataset consisting of image data, meaning that 75% of the data points in the dataset are below this value;

[0101] Let IQR = Q3 - Q1, where IQR is the difference between Q3 and Q1, representing the middle 50% distribution range of the dataset. Then we have a lower bound Lm = Q1 - 1.5 × IQR and an upper bound Hm = Q3 + 1.5 × IQR. When the value of a data point in the dataset is lower than the lower bound Lm or higher than the upper bound Hm, the value of that data point is an outlier.

[0102] Example: The initial dataset is [12,15,14,10,18,20,25,30,35,40,100]. The data points are sorted in ascending order, resulting in a sorted dataset [10,12,14,15,18,20,25,30,35,40,100]. This sorted dataset contains 11 data points. In this sorted dataset, Q1 is located at (11+1) / 4 = 3, which is the third position in the sorted dataset. Therefore, Q1 = the value of the third data point = 14. The position of Q3 is 3 × Q1 = 3 × (11+1) / 4 = 9, which is the ninth position in the sorted dataset. Therefore, Q3 = the value of the ninth data point = 35. Given IQR = Q3 - Q1 = 21, we can further derive the lower bound Lm = Q1 - 1.5 × IQR = 14 - 1.5 × 21 = -17.5, and the upper bound Hm = Q3 + 1.5 × IQR = 35 + 1.5 × 21 = 66.5. In the sorted dataset, only the last data point's value of 100 is higher than the upper bound; the values ​​of the remaining data points are all between the lower bound Lm and the upper bound Hm. Therefore, only the last data point's value is an outlier. In practice, datasets contain at least several hundred data points. When calculating, if there is no data point with a 25th percentile, the closest data point should be used. Outliers in image data must be removed.

[0103] For the three types of data mentioned above—text data, numerical data, and image data—data deduplication is also required: remove duplicate samples from the dataset to ensure that each data point has no duplicate values.

[0104] S2. Process the data processed in step S12 using a multimodal algorithm model. This includes the following steps:

[0105] S21. Use a Convolutional Neural Network (CNN) to preprocess the image data. The core idea of ​​CNN is to automatically extract image features using convolution operations, thereby reducing reliance on manual intervention. This step specifically includes the following steps:

[0106] S211, Image preprocessing.

[0107] Before inputting image data into a CNN, preprocessing is required, including the following steps:

[0108] S2111. Adjust image size: Adjust all images to the same size (e.g., 224×224) to ensure consistent input format. Specifically, this involves scaling the pixel values ​​of the images from [0,225] to [0,1], i.e., pixel normalization. This scales the image data to a more manageable range, helping the model train more stably and process images. The formula is:

[0109] Np = pv / 255,

[0110] In the above formula, pv is the pixel value in the image, and Np is the normalized pixel value.

[0111] S2112, Data Augmentation: Perform random rotation, cropping, flipping, and other operations on images to increase data diversity and prevent overfitting.

[0112] S212. Construct a CNN model. This includes the following steps:

[0113] S212a. Setting up the convolution operation. The convolution operation is the core of CNN. CNN extracts local features from the input image through convolution kernels, or filters. Each convolution kernel is a small matrix (e.g., a 3x3 or 3x5 matrix) that slides across the input image and performs dot product operations to generate a feature map. In the convolution operation, the convolution kernel K is a trainable parameter that affects the final feature map. To further extend this, this invention introduces a learnable scale factor α and a learnable bias term β to make the output of the convolution operation more flexible. The convolution operation formula is as follows:

[0114]

[0115] Let the specific pixel coordinates of the input image be the initial pixel position, and the specific pixel coordinates of the convolution kernel be the kernel pixel position. Then, in the above formula, F(i,j) is the output value at the initial pixel position (i,j) after the convolution operation; I(i+m,j+n) is the pixel value of the input image at the initial pixel position (i+m,j+n); K(m,n) is the weight of the convolution kernel at the kernel pixel positions (m,n); M and N are the height and width of the convolution kernel; the scale factor α is used to adjust the output amplitude of the convolution operation, and the bias term β is used to adjust the baseline value of the convolution operation output. Where i, j, m, and n are integers, and M and N are positive integers.

[0116] S212b. Setting the activation function. The activation function is used to introduce non-linearity, enabling the model to learn complex features. The activation function used in this invention adds a smoothing parameter to the traditional ReLU function to control its smoothness. This reduces the discontinuity of the activation function at x = 0. The traditional ReLU function is: f(x) = max(0,x). The activation function used in this invention is:

[0117]

[0118] In the above formula, α1 is a smoothing parameter, which depends on the specific working condition. For a larger α1 value, the function SReLU(x) will be closer to the traditional ReLU function, but it will be smoother at x=0. The traditional ReLU function directly retains the positive values ​​of the feature map and sets all negative values ​​in the feature map to 0. Therefore, the traditional ReLU function has the following problems: 1. Although nonlinearity is introduced in the positive interval, the discontinuity at x=0 may cause gradient problems, especially in deep networks; 2. The output is zero in the negative region, which may lead to the "dead neuron" problem, that is, some neurons will never be activated during training, thus losing their learning ability. The function SReLU(x) provided by this invention alleviates the discontinuity at x=0 by using the smoothing parameter α, making the activation function smoother, thereby improving gradient propagation and enhancing the training stability and performance of the network. In addition, the function SReLU(x) has a small positive output even in the negative interval, which avoids the negative region being completely zero, helps the network learn more features, and improves the expressive power of the model.

[0119] S212c, Set up pooling operations. Pooling operations are used to reduce the spatial dimensionality of feature values, reduce computational complexity, and reduce overfitting. This invention uses max pooling: a fixed-size pooling window (e.g., 2x2) is used in the feature pool, and the maximum value (i.e., the maximum value among all pixel values ​​in the window) is selected in each window, i.e., max pooling, to reduce the size of the feature map and retain important features in the image. When the input image passes through the convolutional layer of a convolutional neural network, the convolutional kernel scans various regions of the image and generates a new image form by weighted summation with the pixels of the local image region. This new image is the feature map, that is, the feature map refers to the feature representation extracted from the input image through convolution operations.

[0120] When the pooling window slides across the feature map, with a stride of 2, each time the window slides over a portion of the feature map, the maximum pixel value of that region is calculated and recorded.

[0121] Feature maps are typically smaller than the original image size because convolution and pooling operations reduce the image size. To maintain the feature map size the same as the input image size, this invention adds extra pixels to the boundaries of the input image and pads these pixels with zeros.

[0122] S212d, Classification or Regression. Classification or regression combines the extracted features and makes a final prediction. In this step, the multidimensional feature map after convolution and pooling is first flattened into a one-dimensional vector. The flattening operation converts two-dimensional data into one-dimensional data so that it can be input into the fully connected layer. This layer generates new data by multiplying each input data point with a weight and adding a bias value (here, the bias value can be any real number, and it does not have a fixed range; its actual value depends on the initialization method and training process), thereby transforming the feature map data into the final result format.

[0123] S212e, Model Output. If the simulation target is a classification task, the softmax function is used to convert the results of the fully connected layer into probabilities for each class, and the class with the highest probability is output. If the simulation target is a regression task, a continuous numerical result is directly output to represent the specific predicted value.

[0124] S213. The CNN model constructed in step S212 processes the data obtained in step S211. For each image, a fixed-length feature vector is extracted from the last layer (pooling layer) of the CNN model, denoted as the first feature vector. This vector is typically 4096-dimensional. When there are n1 (n1 is an integer greater than 1) images in the multimodal data, the multimodal data processed in step S211 is input into the CNN model to obtain n1 first feature vectors. These n1 first feature vectors are assigned different weights and then summed (i.e., the components corresponding to each vector are added together to obtain the components of the total vector) to obtain a total 4096-dimensional feature vector. This total 4096-dimensional feature vector is denoted as the image feature vector. The weights are set based on the importance of image features in the final task, and are usually adaptively adjusted during the training of the CNN model.

[0125] S22. Use a Recurrent Neural Network (RNN) to preprocess textual data such as articles or comments. An RNN is a neural network structure specifically designed for processing sequential data. RNNs have a "memory" function, allowing them to utilize previous information to influence subsequent calculations, making them very effective in processing time series data and natural language processing tasks. This step includes the following:

[0126] S221. Text preprocessing, including the following steps:

[0127] S221a, Text Cleaning: Remove irrelevant content such as punctuation marks, numbers, and special characters from text.

[0128] S221b. Word Segmentation: The text is segmented into words or phrases to obtain a text sequence. The word segmentation tool used in this invention is jieba, which is a widely used Chinese word segmentation library. For example, the word segmentation result of the sentence "I love learning" is ["I", "love", "learning"].

[0129] S221c. Create Vocabulary: A technician creates a vocabulary containing all words related to used car evaluation and prediction, denoted as the used car evaluation and prediction vocabulary, and assigns a unique index to each word in it. For example, for the three words in the word segmentation result of step S221b, the indices are assigned in sequence: 1, 2, 3, and the corresponding vocabulary can be obtained as {"I": 1, "love": 2, "learning": 3}.

[0130] S221d. Sequence Encoding: Based on the index, the segmented text sequence is correspondingly converted into an index sequence. For example, ["I", "love", "learning"] can be encoded into the index sequence [1, 2, 3].

[0131] S221e. Perform padding or truncating processing on all index sequences to make all sequences have the same length. In this invention, 0 is used for padding.

[0132] S222. Build an RNN model. Use a deep learning framework (such as TensorFlow, pytorch) to build an RNN model, including the following steps:

[0133] S222a. Set up an embedding layer. The main function of the embedding layer is to convert discrete vocabulary indices into continuous dense vector representations, and these vectors capture the semantic information of the vocabulary in a high-dimensional space. Setting up and using the embedding layer includes the input, operations performed, and output of the embedding layer as follows:

[0134] Input of the embedding layer: Index sequence, where each index represents a word.

[0135] Operations performed by the embedding layer: Look up the embedding matrix and convert the index into the corresponding embedding vector in the embedding matrix. The embedding matrix is a predefined matrix that contains the vector information of all words in the vocabulary and is used to convert words from index form into their corresponding embedding vectors.

[0136] Output of the embedding layer: Embedding vector sequence, where each embedding vector represents the high-dimensional semantic representation of a word.

[0137] For example: For a sentence "I love learning", after encoding with the vocabulary, the index sequence [1, 2, 3] is obtained, and the embedding layer will convert this sequence into three corresponding vectors.

[0138] S222b, Setting up the RNN layer. The RNN layer is the core of the model, used to process sequential data. Setting up the RNN layer includes the RNN layer input, the operations performed, and the output, as follows:

[0139] RNN layer input: sequence of embedding vectors.

[0140] The RNN layer performs the following operations: At each time step, the RNN layer accepts the current input and the hidden state from the previous time step, and calculates a new hidden state. The hidden state is the internal state used by the RNN layer to store and transmit information when processing the input sequence. In an RNN, the initial value of the hidden state is set to an all-zero vector. This initial value is used as the starting point of the RNN network at the beginning of the first time step of the hidden state sequence. Definition of the initial hidden state: Assuming the hidden state dimension of the RNN network is h, if processing a batch of data with a batch size of b, the shape of the initial hidden state is (b, h), where b and h are integers greater than 0. The initial hidden state is initialized as an all-zero vector, meaning each element is 0. Specifically, the initial hidden state is a vector consisting of h zeros.

[0141] RNN layer output: a sequence of hidden states, where each hidden state contains information from the beginning of the hidden state sequence to the current time step.

[0142] S222c. Set the activation function. The activation function is mainly used to introduce nonlinear transformations, enabling the model to capture complex patterns. In this step, the activation function can be the previously mentioned ReLU (Rectified LinearUnit): setting negative values ​​in the input to 0, while keeping positive values ​​unchanged. Alternatively, the Sigmoid function can be used to compress the input to between 0 and 1, or the Softmax function can be used to convert the input vector into a probability distribution.

[0143] S223. Using the RNN model constructed in step S222 to process the data obtained in step S221, a segment of continuous text data in the multimodal data can obtain a feature vector of fixed length, denoted as the second feature vector. This vector is usually 256-dimensional. When there are n2 (n2 is an integer greater than 1) segments of continuous text data in the multimodal data, the multimodal data processed in step S221 is input into the RNN model to obtain n2 second feature vectors. These n2 second feature vectors are added together to obtain a total 256-dimensional feature vector, which is denoted as the text feature vector.

[0144] S23. Use a linear regression model to preprocess the numerical data. Linear regression aims to make predictions by establishing a linear relationship between the target variable and the feature variables. This step specifically includes the following steps:

[0145] S231. Numerical data preprocessing, including the following steps:

[0146] S231a. Use data analysis tools such as pandas' isnull function to identify missing values ​​in numerical data. For example, data.isnull().sum() can display the number of missing values ​​in each column.

[0147] S231b: Use methods such as mean, median, and mode to fill in missing data.

[0148] S231c. Identify and handle outliers in the data to reduce their impact on the model. Use the IQR method (see above for a detailed explanation of IQR) to calculate upper and lower bounds for outliers and remove data that exceeds these bounds.

[0149] S232. Construct a linear regression model, including creating a linear regression model instance using the LinearRegression class.

[0150] S233. Using the linear regression model constructed in step S232 to process the data obtained in step S231, a fixed-length feature vector, denoted as the third feature vector, can be obtained for each numerical data point in the multimodal data. This vector is typically 64-dimensional. When there are n3 (n3 is an integer greater than 1) numerical data points in the multimodal data, the multimodal data processed in step S231 is input into the linear regression model to obtain n3 third feature vectors. These n3 third feature vectors are added together to obtain a total 64-dimensional feature vector, which is denoted as the numerical feature vector.

[0151] S3. Based on step S2, perform modal fusion processing to combine information from different sensory data (such as images, text, and sound) so that the computer can more comprehensively understand and process used car-related data, i.e., multimodal data.

[0152] Specifically, modal fusion includes the following steps:

[0153] S31. Data Integration. The image feature vector, text feature vector, and numerical feature vector are concatenated into a comprehensive feature vector. If the image feature vector is 4096-dimensional, the text feature vector is 256-dimensional, and the numerical feature vector is 64-dimensional, then the comprehensive feature vector is 4416-dimensional.

[0154] S32. Configure a fully connected neural network (DNN) model. This includes:

[0155] Set up an input layer with the same size as the combined feature vector, to receive input data such as feature vectors.

[0156] A hidden layer is set up, with at least one hidden layer added after the input layer. Each hidden layer has, for example, 512 neurons, using the ReLU activation function. Each neuron receives the input data and linearly transforms it using weights and biases.

[0157] The output layer is configured with different numbers of neurons and activation functions depending on whether the task is classification or regression. For binary classification tasks (in the used car industry, binary classification is used to make simple, binary judgments about vehicles, such as accident vehicle identification and warranty status checks. These tasks share a common characteristic: they involve classifying vehicles into two mutually exclusive categories, such as "qualified / unqualified" or "present / absent." Binary classification models are particularly suitable for these tasks), the output layer may have one neuron, using the sigmoid activation function. For multi-class classification tasks, the number of neurons in the output layer equals the number of categories, with each neuron corresponding to one category. The output layer uses the softmax activation function, which transforms the output value of each neuron into a probability between 0 and 1 (i.e., the probability of the corresponding task being accomplished), ensuring that the sum of all output probabilities is 1.

[0158] In used car-related tasks, the output of the output layer depends on the specific business requirements, and can be divided into the following three categories:

[0159] S32a. Predicting Used Car Prices (Regression Task): If the task is to predict the selling price of a used car, the output layer will have one neuron. The neuron uses a linear activation function such as the Sigmoid activation function described above to output a continuous value, which represents the predicted used car price.

[0160] S32b, Predicting Used Car Categories (Classification Task): If the task is to classify the type of used car (such as sedan, SUV, truck, etc.), the output layer will have more than one neuron (each category corresponds to one neuron). Each neuron uses the linear activation function to output a probability distribution, where the probability represents the probability that the input used car belongs to a different type.

[0161] S32c, Used Car Quality Classification: If the task is to evaluate the quality of used cars, and the quality is divided into 3 levels: "Good", "Medium" and "Poor", then the output layer has 3 neurons. Each neuron also uses a linear activation function to output 3 probability values, representing the probability that the used car belongs to each quality level.

[0162] This invention achieves comprehensive evaluation and accurate prediction of used car value by integrating image, text, and numerical data. Image data is processed using a convolutional neural network to extract features, text data is processed using a natural language processing model, and numerical data is standardized and used directly. These features are concatenated or weighted into a unified feature vector, which is then input into a comprehensive model (such as a deep neural network or a Transformer model). This invention not only improves the accuracy of used car market value assessment but also provides personalized price predictions and quality analysis, helping users make more informed purchasing decisions.

[0163] This invention designs a series of efficient multimodal algorithm models. These models not only effectively process complex multimodal data but also improve operational efficiency while maintaining high accuracy, enabling the invention to perform exceptionally well in handling complex data and significantly enhancing evaluation and prediction performance. Specifically, these models can comprehensively process multiple data types such as images, text, and numerical data. Through innovative algorithms and optimized model structures, they can deeply mine the deep features within the data. They not only excel in accuracy but also significantly improve operational efficiency, substantially reducing computational overhead and processing time. This efficiency allows the models to perform exceptionally well in real-time data processing and large-scale datasets. The models of this invention possess high adaptability, flexibly handling various complex real-world application scenarios, providing accurate and useful results in areas such as used car market value assessment, personalized price prediction, and quality analysis. Through precise feature extraction and intelligent fusion, these models significantly improve the overall performance of evaluation and prediction, helping users make more informed decisions. In summary, this invention demonstrates superior performance in processing complex multimodal data, enhancing comprehensive evaluation and prediction capabilities.

[0164] Furthermore, this invention reduces manual intervention and improves work efficiency and consistency.

[0165] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for used car evaluation and prediction based on a multimodal algorithm model, characterized in that, Including the following steps: S1. Multimodal data collection and preprocessing; The multimodal data collection includes: collecting used car-related data from multiple channels, i.e., multimodal data, which includes numerical data, image data, and text data; wherein, the numerical data includes vehicle price, mileage, and year, the image data includes photos of the vehicle exterior, and the text data includes user reviews and vehicle condition descriptions; S2. Process the data processed in step S1 using a multimodal algorithm model, including: using a convolutional neural network to preprocess image data to obtain image feature vectors; using a recurrent neural network to preprocess text data to obtain text feature vectors; and using a linear regression model to preprocess numerical data to obtain numerical feature vectors. S3. Based on step S2, perform modal fusion processing, including: Data integration: The image feature vector, text feature vector, and numerical feature vector are concatenated into a comprehensive feature vector; Setting up a fully connected neural network model involves the following steps: Set an input layer whose size is the same as the length of the synthesized feature vector; A hidden layer is set up, and at least one hidden layer is added after the input layer. After receiving the input data, the neurons in each hidden layer linearly transform the data by weights and biases. An output layer is set up. In used car-related tasks, the output results of the output layer depend on the specific business requirements and are divided into the following three categories: Predicting Used Car Prices: If the task is to predict the selling price of a used car, the output layer has one neuron that uses a linear activation function to output a continuous value, which represents the predicted used car price. Predicting the type of used car: If the task is to classify the type of used car, the output layer has multiple neurons, and each type corresponds one-to-one with one of the multiple neurons. An activation function is used to output a probability distribution, where the probability represents the probability that the input used car belongs to a different type. Used car quality classification: If the task is to evaluate the quality of a used car, and the quality is divided into three levels: good, medium, and poor, then the output layer has three neurons, which use activation functions to output three probability values, representing the probability that the used car belongs to each quality level.

2. The used car evaluation and prediction method based on a multimodal algorithm model according to claim 1, characterized in that, Step S1 includes: For text data, remove punctuation marks and standardize capitalization to clean and standardize the text content; For numerical data, fill in the missing values ​​with the mean, median, or mode of the values ​​other than the missing values; For image data, i.e., pixel values, we remove noise, adjust brightness and size, and improve image quality. Here, we use filters to smooth the image and remove noise. For the text data, numerical data, and image data, perform data deduplication: delete duplicate samples in the dataset to ensure that each data point has no duplicate values.

3. The used car evaluation and prediction method based on a multimodal algorithm model according to claim 2, characterized in that, The filter is a Gaussian filter or a median filter.

4. A used car evaluation and prediction method based on a multimodal algorithm model according to claim 2 or 3, characterized in that, Step S1 further includes the following step: S1a. Outliers in the image data are processed as follows: Outliers are detected using the quartile range method. A quartile is a statistical measure that divides a dataset into four equal parts. Specifically, the quartile range method includes: Let Q1 be the first quartile, representing the 25th percentile of the dataset consisting of the image data, that is, 25% of the data points in the dataset are below this value; Let Q3 be the third quartile, representing the 75th percentile of the dataset consisting of the image data, that is, 75% of the data points in the dataset are below this value; Let IQR = Q3 - Q1, where IQR is the difference between Q3 and Q1, representing the middle 50% distribution range of the dataset. Then, we have a lower bound Lm = Q1 - 1.5 × IQR and an upper bound Hm = Q3 + 1.5 × IQR. When the value of a data point in the dataset is lower than the lower bound Lm or higher than the upper bound Hm, the value of the data point is an outlier, and the outlier is deleted.

5. The used car evaluation and prediction method based on a multimodal algorithm model according to claim 4, characterized in that, In step S2, The image data is preprocessed using a convolutional neural network, including the following steps: S211. Image preprocessing, including the following steps: Adjust image size: Resize all images to the same size to ensure consistent input format. Specifically, scale the pixel values ​​of the images from [0, 225] to [0, 1], i.e., pixel normalization. The formula is: Np = pv / 255. In the above formula, pv is the pixel value in the image, and Np is the normalized pixel value; Data augmentation: Randomly rotate, crop, and flip the image. S212. Construct a convolutional neural network model, including the following steps: The convolution operation is set up using the following formula: Let the specific pixel coordinates of the input image be the initial pixel position, and the specific pixel coordinates of the convolution kernel be the convolution kernel pixel position. Then, in the above formula, F(i,j) is the output value at the initial pixel position (i,j) after the convolution operation; I(i+m,j+n) is the pixel value of the input image at the initial pixel position (i+m,j+n). K(m,n) is the weight of the convolution kernel at pixel position (m,n); M and N are the height and width of the convolution kernel; the scale factor α is used to adjust the output amplitude of the convolution operation, and the bias term β is used to adjust the baseline value of the output of the convolution operation, where i, j, m, n are integers, and M and N are positive integers. Set the activation function: ，（1） In the above formula, α1 is a smoothing parameter, which depends on the specific working conditions. A pooling operation is configured, using a fixed-size pooling window in the feature pool. The maximum value is selected within each window, i.e., max pooling. To maintain the feature map size matching the input image size, extra pixels are added to the boundaries of the input image and padded with zeros. In classification or regression, this step first flattens the multidimensional feature map after convolution and pooling into a one-dimensional vector. This flattening operation transforms two-dimensional data into one-dimensional data, which is then input into the fully connected layer. The fully connected layer generates new data by multiplying each input data point by a weight and adding a bias value, thus transforming the feature map data into the final result format. The model's output varies depending on the simulation task. If the simulation is for classification, the softmax function is used to convert the results of the fully connected layers into probabilities for each class, and the class with the highest probability is output. If the simulation is for regression, a continuous numerical result is directly output to represent the predicted value. The convolutional neural network model constructed in step S212 processes the data obtained in step S211. A fixed-length feature vector, denoted as the first feature vector, is extracted from the last layer (pooling layer) of the convolutional neural network model for each image. When there are n1 images in the multimodal data, n1 is an integer greater than 1. After processing the multimodal data in step S211, the data is input into the convolutional neural network model to obtain n1 first feature vectors. The n1 first feature vectors are assigned different weights and then summed to obtain a total feature vector, denoted as the image feature vector. The weights are set based on the importance of image features in the final task.

6. The used car evaluation and prediction method based on a multimodal algorithm model according to claim 5, characterized in that, In step S2, The text data is preprocessed using a recurrent neural network, including the following steps: S221. Text preprocessing, including the following steps: Text cleaning: Remove punctuation marks, numbers, and special characters from the text; Word segmentation: The text is divided into words or phrases to obtain a text sequence; Create a vocabulary: Create a vocabulary for used car evaluation and prediction, and assign a unique index to each word in it; Sequence encoding: Based on the index, the text sequence is correspondingly converted into an index sequence; All the index sequences are padded or truncated to ensure they have the same length. S222. Construct a recurrent neural network model, including the following steps: Set up an embedding layer: The input to the embedding layer is a sequence of indexes, where each index represents a word; The operations performed by the embedding layer are: looking up the embedding matrix and converting the index into the corresponding embedding vector; the output of the embedding layer is a sequence of embedding vectors, where each embedding vector represents a high-dimensional semantic representation of the word. Set up a recurrent neural network layer: The input of the recurrent neural network layer is the embedding vector sequence; The operation performed by the recurrent neural network layer is: At each time step, the recurrent neural network layer accepts the current input and the hidden state of the previous time step, and calculates a new hidden state; The output of the recurrent neural network layer is a hidden state sequence, wherein each hidden state contains information from the beginning of the hidden state sequence to the current time step; Set the activation function: Use ReLU (Rectified LinearUnit): set negative values ​​in the input to 0, and leave positive values ​​unchanged; or use the Sigmoid function to compress the input to between 0 and 1; or use the Softmax function to convert the input vector into a probability distribution; S223. The data obtained in step S221 is processed by the recurrent neural network model. A segment of continuous text data in the multimodal data is given a feature vector of fixed length, denoted as the second feature vector. When there are n2 segments of continuous text data in the multimodal data, n2 is an integer greater than 1. The multimodal data is processed by step S221 and then input into the recurrent neural network model to obtain n2 second feature vectors. The n2 second feature vectors are added together to obtain a total feature vector, denoted as the text feature vector.

7. The used car evaluation and prediction method based on a multimodal algorithm model according to claim 6, characterized in that, In step S2, The numerical data is preprocessed using a linear regression model, including the following steps: S231. Numerical data preprocessing, including the following steps: Identify missing values ​​in the numerical data; Fill in the missing data; Identify and process outliers in the numerical data. S232. Construct a linear regression model, including creating a linear regression model instance using the LinearRegression class. S233. The linear regression model constructed in step S232 is used to process the data obtained in step S231. In the multimodal data, a fixed-length feature vector is obtained for each numerical data point, denoted as the third feature vector. When there are n3 numerical data points in the multimodal data, n3 is an integer greater than 1. The multimodal data is processed by step S231 and then input into the linear regression model to obtain n3 third feature vectors. The n3 third feature vectors are added together to obtain a total feature vector, denoted as the numerical feature vector.

8. The used car evaluation and prediction method based on a multimodal algorithm model according to claim 7, characterized in that, In step S231, the quartile range method is used to process and calculate the upper and lower bounds of outliers in the numerical data. That is, after replacing the image data in step S1a with the numerical data, step S1a is executed again, and then the numerical data that exceeds the upper and lower bounds is removed.

Images