Second-hand car evaluation and prediction method based on multi-modal algorithm model

Abstract

The invention provides a second-hand car evaluation and prediction method based on a multi-modal algorithm model. The method comprises the following steps: S1, collecting and preprocessing multi-modal data; s2, processing the data processed in the step S1 by adopting a multi-modal algorithm model; and S3, performing modal fusion processing based on the step S2. According to the method, complex multi-modal data can be effectively processed, the operation efficiency of the model can be improved while high accuracy is kept, the method is excellent in performance when complex data are processed, the evaluation and prediction performance is greatly improved by integrating images, texts and numerical data, and the method is suitable for popularization and application. And comprehensive evaluation and accurate prediction of the second-hand car are realized. In addition, personalized price prediction and quality analysis can be provided, and the user is helped to make a more intelligent purchase decision.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle detection and analysis, and in particular relates to a second-hand car evaluation and prediction method based on a multimodal algorithm model. Background Art

[0002] With the rapid development of the used car market, consumers' demand for used cars continues to increase, and the scale of the used car market is also expanding year by year. However, due to the complex sources and different conditions of used cars, it is extremely important to accurately evaluate the value of used cars and predict market demand. Traditional used car evaluation methods mainly rely on manual experience and simple statistical models, which are difficult to accurately reflect the condition of the car and market dynamics, and cannot meet the needs of the modern market.

[0003] At present, data science and artificial intelligence technologies are increasingly being used in various fields, especially in big data processing and analysis, and have achieved remarkable results. In the field of used cars, the use of big data and artificial intelligence technologies can achieve more accurate evaluation and prediction. Although some existing implementation solutions, such as used car evaluation methods based on machine learning algorithms such as linear regression and decision trees, have improved accuracy to a certain extent, there are still many shortcomings:

[0004] Insufficient data utilization: Existing methods mainly rely on single-modality data (such as numerical data), ignoring other important information such as images and texts, resulting in inaccurate evaluation results.

[0005] Limited model performance: Traditional machine learning models such as linear regression and decision trees have limited performance when processing complex used car data and cannot fully capture the deep features in the data.

[0006] High computing resource requirements: Some complex models (such as deep learning models) require a lot of computing resources during training and prediction, resulting in high costs and not suitable for promotion in practical applications.

[0007] Extensive manual involvement: Many existing methods still require a lot of manual involvement, especially in the data processing and feature extraction stages, which is inefficient and prone to human errors. Summary of the invention

[0008] In view of the above problems, the present invention provides a used car evaluation and prediction method based on a multimodal algorithm model.

[0009] The present invention provides a second-hand car evaluation and prediction method based on a multimodal algorithm model, comprising the steps of:

[0010] S1, multimodal data collection and preprocessing;

[0011] S2, using a multimodal algorithm model to process the data processed in step S1;

[0012] S3. Perform modal fusion processing based on step S2.

[0013] further,

[0014] The step S1 comprises:

[0015] For text data, remove punctuation and unify capitalization to clean and standardize the text content;

[0016] For numerical data, the missing values ​​are filled with the mean, median or mode of the values ​​other than the missing values;

[0017] For image data, i.e. pixel values, remove noise, adjust brightness and size, and improve image quality. Here, filters are used to smooth the image and remove noise;

[0018] For the text data, numerical data and image data, data deduplication is performed: duplicate samples in the data set are deleted to ensure that each data has no duplicate values.

[0019] further,

[0020] The filter is a Gaussian filter or a median filter.

[0021] further,

[0022] The step S1 further comprises the steps of:

[0023] S1a. The outliers of the image data are processed as follows: the outliers are detected using the interquartile range method. The quartile is a statistic that divides a data set into four equal parts. Specifically, the interquartile range method includes:

[0024] Q1 is the first quartile, representing the 25th percentile of the data set consisting of the image data, that is, 25% of the data points of the data set are below this value;

[0025] Q3 is the third quartile, representing the 75th percentile of the data set consisting of the image data, that is, 75% of the data points of the data set are below this value;

[0026] Let IQR = Q3-Q1, where IQR is the difference between Q3 and Q1, representing the distribution range of the middle 50% of the data set. Then there is a lower bound Lm = Q1-1.5×IQR, and an upper bound Hm = Q3+1.5×IQR. When the value of a data point in the data set is lower than the lower bound Lm, or higher than the upper bound Hm, the value of the data point is an outlier and the outlier is deleted.

[0027] further,

[0028] In step S2,

[0029] The convolutional neural network is used to pre-process the image data, including the following steps:

[0030] S211, image preprocessing, including the following steps:

[0031] Adjust the image size: adjust all images to the same size to ensure consistent input format. Specifically, the pixel values ​​of the images are scaled from [0,225] to [0,1], i.e., pixel normalization. The formula is: Np = pv / 255.

[0032] In the above formula, pv is the pixel value in the picture, and Np is the normalized pixel value;

[0033] Data enhancement: Randomly rotate, crop, and flip the images.

[0034] S212, constructing a convolutional neural network model, comprising the following steps:

[0035] Set the convolution operation, the formula is as follows:

[0036]

[0037] Let the specific pixel coordinate point of the input image be the initial pixel position, and the specific pixel coordinate point in the convolution kernel be the convolution kernel pixel position. In the above formula, F(i,j) is the output value at the initial pixel position (i,j) after the convolution operation; I(i+m,j+n) is the pixel value of the input image at the initial pixel position (i+m,j+n); K(m,n) is the weight of the convolution kernel at position (convolution kernel pixel)m,n); M and N are the height and width of the convolution kernel; the scale factor α is used to adjust the output amplitude of the convolution operation, and the bias term β is used to adjust the reference value of the convolution operation output, where i, j, m, n are integers, M and N are positive integers,

[0038] Set the activation function:

[0039]

[0040] In the above formula, α1 is a smoothing parameter, which depends on the specific working conditions.

[0041] Set the pooling operation, use a fixed-size pooling window to perform the pooling operation on the feature pool, select the maximum value in each window, that is, the maximum pooling; in order to keep the size of the feature map the same as the size of the input image, add additional pixels at the boundary of the input image and fill the pixels with 0,

[0042] Classification or regression, in this step, the multi-dimensional feature map after convolution and pooling is first flattened into a one-dimensional vector. The flattening operation converts the two-dimensional data into one-dimensional data so that it can be input into the fully connected layer. The fully connected layer generates new data by multiplying each input data point with a weight and adding a bias value, thereby converting the feature map data into the final result format.

[0043] The output of the model, if the simulation calculation target is a classification task, use the softmax function to convert the results of the fully connected layer into the probability of each category, and output the category with the highest probability. If the simulation calculation target is a regression task, directly output a continuous numerical result to represent the specific value of the prediction.

[0044] The convolutional neural network model constructed in step S212 processes the data processed in step S211, and a feature vector of fixed length is extracted from the last layer, i.e., the pooling layer, of the convolutional neural network model for the data of one of the pictures, and recorded as the first feature vector. When there are n1 pieces of data of the pictures in the multimodal data, n1 is an integer greater than 1. The multimodal data is input into the convolutional neural network model after being processed in step S211 to obtain n1 first feature vectors, and different weights are assigned to the n1 first feature vectors, and then the sum is obtained to obtain a total feature vector, which is recorded as the picture feature vector, wherein the weight is set based on the importance of the image features in the final task.

[0045] further,

[0046] In step S2,

[0047] The recurrent neural network is used to pre-process the text data, including the following steps:

[0048] S221, text preprocessing, including the following steps:

[0049] Text cleaning: removing punctuation marks, numbers, and special characters from the text;

[0050] Word segmentation: segment the text into words or characters to obtain a text sequence;

[0051] Create vocabulary: Create a used car evaluation and prediction vocabulary and assign a unique index to each word in it;

[0052] Sequence encoding: based on the index, convert the text sequence into an index sequence accordingly;

[0053] padding or truncating all the index sequences so that all the index sequences have the same length,

[0054] S222, constructing a recurrent neural network model, comprising the following steps:

[0055] Setting an embedding layer: the input of the embedding layer is an index sequence, where each index represents a word; the operation performed by the embedding layer is: looking up the embedding matrix and converting the index into a corresponding embedding vector; the output of the embedding layer is an embedding vector sequence, where each embedding vector represents a high-dimensional semantic representation of the word;

[0056] A recurrent neural network layer is set: the input of the recurrent neural network layer is the embedding vector sequence; the operation performed by the recurrent neural network layer is: at each time step, the recurrent neural network layer accepts the current input and the hidden state of the previous time step, and calculates a new hidden state; the output of the recurrent neural network layer is a hidden state sequence, wherein each hidden state contains information from the beginning of the hidden state sequence to the current time step;

[0057] Set the activation function: Same as formula (1),

[0058] S223. Process the data obtained by processing step S221 with the recurrent neural network model, then a continuous text data in the multimodal data obtains a feature vector of fixed length, which is recorded as the second feature vector. When there are n2 continuous text data in the multimodal data, n2 is an integer greater than 1. The multimodal data is input into the recurrent neural network model after being processed in step S221 to obtain n2 second feature vectors. The n2 second feature vectors are added together to obtain a total feature vector, which is recorded as the text feature vector.

[0059] further,

[0060] In step S2,

[0061] The linear regression model is used to pre-process the numerical data, including the following steps:

[0062] S231, numerical data preprocessing, including the following steps:

[0063] identifying missing values ​​of the numerical data;

[0064] Fill in missing data;

[0065] identifying and processing outliers in said numerical data,

[0066] S232, build a linear regression model, including using the LinearRegression class to create a linear regression model instance,

[0067] S233. Process the data obtained by processing in step S231 with the linear regression model constructed in step S232, and obtain a eigenvector of fixed length for one numerical data in the multimodal data, which is recorded as the third eigenvector. When there are n3 numerical data in the multimodal data, n3 is an integer greater than 1. The multimodal data processed in step S231 is input into the linear regression model to obtain n3 third eigenvectors. The n3 third eigenvectors are added together to obtain a total eigenvector, which is recorded as the numerical eigenvector.

[0068] further,

[0069] In the step S231, the interquartile range method is used to calculate the upper and lower limits of the abnormal values ​​in the numerical data, that is, after replacing the image data in the step S1a with the numerical data, the step S1a is executed again, and then the numerical data exceeding the upper and lower limits are removed.

[0070] further,

[0071] The step S3 comprises the following steps:

[0072] Data integration: splicing the image feature vector, text feature vector and numerical feature vector into a comprehensive feature vector;

[0073] Setting up a fully connected neural network model involves the following steps:

[0074] Setting an input layer, whose size is the same as the length of the comprehensive feature vector;

[0075] Setting a hidden layer, adding at least one hidden layer after the input layer, after the neurons in each hidden layer receive the input data, they perform a linear transformation on the data through weights and biases;

[0076] Set the output layer. In the used car related tasks, the output results of the output layer depend on the specific business needs and are divided into the following three categories:

[0077] Predicting the price of a used car: If the task is to predict the selling price of a used car, the output layer is provided with a neuron, using a linear activation function to output a continuous value, which represents the predicted price of the used car;

[0078] Predicting the category of a used car: If the task is to classify the type of a used car, the output layer is provided with a plurality of neurons, each type corresponds to one of the plurality of neurons, and an activation function is used to output a probability distribution, wherein the probability represents the probability that the input used car belongs to a different type;

[0079] Used car quality classification: If the task is to evaluate the quality of a used car, and the quality is divided into three quality levels: good, medium, and poor, then the output layer is provided with three neurons, which use activation functions to output three probability values, indicating the probability of a used car belonging to each quality level.

[0080] The second-hand car evaluation and prediction method based on the multimodal algorithm model provided by the present invention designs a series of efficient multimodal algorithm models, and realizes the comprehensive evaluation and accurate prediction of second-hand cars by integrating images, texts and numerical data. These models can not only effectively process complex multimodal data, but also improve the operating efficiency of the model while maintaining high accuracy. The present invention performs well in processing complex data and greatly improves the performance of evaluation and prediction. The present invention not only improves the accuracy of the evaluation of the market value of used cars, but also provides personalized price prediction and quality analysis to help users make more informed purchasing decisions. In addition, the present invention reduces manual intervention and improves work efficiency and consistency.

[0081] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0083] Figure 1 A flow chart of a used car evaluation and prediction method based on a multimodal algorithm model according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions. The terms "first", "second", "third", etc. in the specification and claims of this application or the above-mentioned figures are used to distinguish different objects, not to describe a specific order or a primary and secondary relationship. "Multiple" appearing in this application refers to more than two (including two).

[0086] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0087] Figure 1 A flow chart of the second-hand car evaluation and prediction method based on a multimodal algorithm model provided by the present invention. The second-hand car evaluation and prediction method based on a multimodal algorithm model includes the following steps, wherein each model mentioned below needs to be trained, verified and tested to complete the evaluation and optimization of each model. The training, verification and testing, as well as the further evaluation and optimization methods are known to those skilled in the art and are not the focus of this application, and therefore will not be described in detail in this application.

[0088] S1. Multimodal data collection and preprocessing.

[0089] Multimodal data collection: Used car related data, i.e. multimodal data, are collected from multiple channels, including vehicle history data, market transaction data, vehicle pictures, and user evaluation texts. These data belong to different modes such as text, images, and audio. The multiple channels include online used car trading platforms, car auction houses, dealers, market surveys, third-party data providers, social media, and discussions and user evaluation channels on online car enthusiast forums. The related data includes numerical data such as vehicle price and mileage, image data such as vehicle appearance photos, and text data such as user evaluation and vehicle condition description. Among them, the used car related data, i.e. multimodal data, requires at least tens of thousands of records of numerical data to capture changes in features such as vehicle price, year, and mileage; image data requires at least thousands of high-quality vehicle photos, covering different brands, models, and vehicle conditions; and text data should contain tens of thousands of comments to cover different users' views and description methods. And it is necessary to prepare multimodal data for implementing the model, which is divided into training sets, validation sets, and test sets for training, validation, and testing the following models to complete the evaluation and optimization of each model.

[0090] Multimodal data preprocessing: Clean, standardize, and enhance the different modes of multimodal data, such as text, images, and audio, to ensure data quality and consistency. Cleaning includes the following steps:

[0091] S11. For text data, remove punctuation and unify capitalization to clean and standardize the text content. For example, the original text is "Hello, world! This is a test, data cleaning. We need to clean the data: remove punctuation; unify capitalization.", and the cleaned text is "Hello world, this is a test, data cleaning, we need to clean the data, remove punctuation, unify capitalization".

[0092] S12. For numerical data, use the mean, median or mode of the values ​​other than the missing values ​​to fill in the missing values. For example, the following Table 1 provides three variables: variable A, variable B, variable C. Each variable

[0093] Table 1

[0094] VariableA Variable B Variable C 10 20 NaN 15 NaN 25 NaN 30 35 40 45 50

[0095] There are 4 possible values, but one of them is missing, and the missing value is recorded as NaN. If the mean of the values ​​other than the missing value is taken to fill the missing value, the mean of the values ​​other than the missing value of variable A is (10+15+40) / 3=65 / 3≈21.67. Similarly, the mean of the values ​​other than the missing value of variable B and variable C is 31.67 and 36.67 respectively. After filling the missing values ​​in Table 1, Table 2 is obtained.

[0096] Table 2

[0097] VariableA Variable B Variable C 10 20 36.67 15 31.67 25 21.67 30 35 40 45 50

[0098] S13. For image data, i.e., pixel values, remove noise, adjust brightness and size, and improve image quality. Here, filters such as Gaussian filters and median filters are used to smooth the image and remove noise. The filter is a tool for removing noise or highlighting certain features in signal processing and image processing. The outliers of the image data are processed as follows: the interquartile range (IQR) method is used to detect outliers. The quartile is a statistic that divides the data set into four equal parts. Specifically, the IQR method includes:

[0099] Q1 is the first quartile, representing the 25th percentile of the data set consisting of image data, that is, 25% of the data points of the data set are below this value;

[0100] Q3 is the third quartile, representing the 75th percentile of the data set consisting of image data, that is, 75% of the data points of the data set are below this value;

[0101] Let IQR = Q3-Q1, where IQR is the difference between Q3 and Q1, representing the distribution range of the middle 50% of the data set. Then there is a lower bound Lm = Q1-1.5×IQR, and an upper bound Hm = Q3+1.5×IQR. When the value of a data point in the data set is lower than the lower bound Lm, or higher than the upper bound Hm, the value of the data point is an outlier.

[0102] For example, the initial data set is [12, 15, 14, 10, 18, 20, 25, 30, 35, 40, 100]. The values ​​of the data points are sorted from small to large, thus obtaining a sorted data set [10, 12, 14, 15, 18, 20, 25, 30, 35, 40, 100]. The number of data points in the sorted data set is 11. For the sorted data set, the position of Q1 is (11+1) / 4=3, which is the third position in the sorted data set, so Q1=the value of the third data point=14. The position of Q3 is 3×Q1=3×(11+1) / 4=9, which is the ninth position in the sorted data set, so Q3=the value of the ninth data point=35. And IQR=Q3-Q1=21, further we can get the lower bound Lm=Q1-1.5×IQR=14-1.5×21=-17.5, and the upper bound Hm=Q3+1.5×IQR=35+1.5×21=66.5. In the sorted data set, only the value of the last data point 100 is higher than the upper bound, and the values ​​of the remaining data points are between the lower bound Lm and the upper bound Hm. Therefore, only the value of the last data point is an outlier. In actual work, the data set contains at least hundreds of data points. When calculating, when taking the 25th percentile data point, if there is no 25th percentile data point, take the data point closest to the 25th percentile. For abnormal values ​​of image data, they need to be deleted.

[0103] For the above three types of data: text data, numerical data and image data, data deduplication is also required: delete duplicate samples in the data set to ensure that each data has no duplicate values.

[0104] S2, using a multimodal algorithm model to process the data processed in step S12. The steps include:

[0105] S21. Use convolutional neural network (CNN) to pre-process the image data. The core idea of ​​CNN is to use convolution operation to automatically extract image features, thereby reducing dependence on manual work. This step specifically includes the following steps:

[0106] S211: Image preprocessing.

[0107] Before inputting the image data into CNN, it needs to be preprocessed, including the following steps:

[0108] S2111. Adjust image size: adjust all images to the same size (such as 224×224) to ensure consistent input format. Specifically, the pixel values ​​of the images are scaled from [0,225] to [0,1], i.e., pixel normalization. This can scale image data to a range that is easy to handle, to help the model train and process images more stably. The formula is:

[0109] Np=pv / 255,

[0110] In the above formula, pv is the pixel value in the image, and Np is the normalized pixel value.

[0111] S2112. Data enhancement: Randomly rotate, crop, flip, and perform other operations on images to increase data diversity and prevent overfitting.

[0112] S212, constructing a CNN model. This includes the following steps:

[0113] S212a, set the convolution operation. The convolution operation is the core of CNN, which extracts local features of the input image through convolution kernels or filters. Each convolution kernel is a small matrix (for example, a 3x3 or 3x5 matrix) that slides on the input image and performs a dot product operation to generate a feature map. In the convolution operation, the convolution kernel K is a trainable parameter that affects the final feature map. For further expansion, the present invention introduces a learnable scale factor α and a learnable bias term β to make the output of the convolution operation more flexible. The convolution operation formula is as follows:

[0114]

[0115] The specific pixel coordinate point of the input image is the initial pixel position, and the specific pixel coordinate point in the convolution kernel is the convolution kernel pixel position. In the above formula, F(i,j) is the output value at the initial pixel position (i,j) after the convolution operation; I(i+m,j+n) is the pixel value of the input image at the initial pixel position (i+m,j+n); K(m,n) is the weight of the convolution kernel at the convolution kernel pixel position)m,n); M and N are the height and width of the convolution kernel; the scale factor α is used to adjust the output amplitude of the convolution operation, and the bias term β is used to adjust the reference value of the convolution operation output. Among them, i, j, m, n are integers, and M and N are positive integers.

[0116] S212b, set the activation function. The activation function is used to introduce nonlinearity so that the model can learn complex features. The activation function used in the present invention is to add a smoothing parameter to the traditional ReLU function to control the smoothness of the activation function. It can reduce the discontinuity of the activation function at x=0. The traditional ReLU function is: f(x)=max(0,x), and the activation function used in the present invention is:

[0117]

[0118] In the above formula, α1 is a smoothing parameter. Depending on the specific working conditions, for larger α1 values, the function SReLU(x) will be closer to the traditional ReLU function, but will be smoother at x=0. The traditional ReLU function directly retains the positive values ​​of the feature map and sets all negative values ​​in the feature map to 0. Therefore, the traditional ReLU function has the following problems: 1. Although nonlinearity is introduced in the positive interval, the breakpoint at x=0 may cause gradient problems, especially in deep networks; 2. The output of the negative area is zero, which may lead to the "dead neuron" problem, that is, some neurons will never be activated during training, thus losing the ability to learn. The function SReLU(x) provided by the present invention alleviates the discontinuity at x=0 through the smoothing parameter α, making the activation function smoother, thereby improving the gradient transfer and enhancing the training stability and performance of the network. In addition, the function SReLU(x) has a small positive output even in the negative interval, which avoids the negative area being completely zero, helps the network learn more features, and improves the expression ability of the model.

[0119] S212c, set the pooling operation. The pooling operation is used to reduce the spatial dimension of the eigenvalue, reduce the computational complexity, and reduce overfitting. The present invention adopts maximum pooling: a fixed-size pooling window (such as 2x2) is used to perform a pooling operation on the feature pool, and the maximum value (that is, the maximum value of all pixel values ​​in the window) is selected in each window, that is, maximum pooling, to reduce the size of the feature map and retain important features in the image. When the input image passes through the convolution layer of the convolutional neural network, the convolution kernel scans various areas of the image, and generates a new image form by weighted summation with the pixels in the local area of ​​the image. This new image is the feature map, that is, the feature map refers to the feature representation extracted from the input image by the convolution operation.

[0120] When the pooling window slides on the feature map, the stride is set to 2. Each time the window slides over a part of the feature map, the maximum pixel value in the area is calculated and recorded.

[0121] The feature map is usually smaller than the original image because the convolution and pooling operations reduce the size of the image. In order to keep the size of the feature map the same as the size of the input image, the present invention adds extra pixels at the border of the input image and fills the pixels with 0.

[0122] S212d, classification or regression. Classification or regression combines the extracted features and makes a final prediction. In this step, the present invention first flattens the multidimensional feature map after convolution and pooling into a one-dimensional vector. The flattening operation converts the two-dimensional data into one-dimensional data so that it can be input into the fully connected layer. This layer generates new data by multiplying each input data point with a weight and adding a bias value (here the bias value can be any real number, which has no fixed range itself, and its actual value depends on the initialization method and training process), thereby converting the feature map data into the final result format.

[0123] S212e, model output. If the simulation calculation target is a classification task, use the softmax function to convert the results of the fully connected layer into the probability of each category, and output the category with the highest probability. If the simulation calculation target is a regression task, directly output a continuous numerical result to represent the specific value of the prediction.

[0124] S213, the CNN model constructed in step S212 processes the data obtained by processing step S211, and extracts a fixed-length feature vector from the last layer (pooling layer) of the CNN model for the data of a picture, which is recorded as the first feature vector, and the vector is usually 4096-dimensional. When there are n1 (n1 is an integer greater than 1) pictures in the multimodal data, the multimodal data is input into the CNN model after being processed in step S211 to obtain n1 first feature vectors, and these n1 first feature vectors are assigned different weights and added (that is, the components corresponding to each vector are added to obtain the components of the total vector) to obtain a total 4096-dimensional feature vector. This total 4096-dimensional feature vector is recorded as the picture feature vector. Among them, the weight is set based on the importance of the image features in the final task, and the weight is usually adaptively adjusted during the training of the CNN model.

[0125] S22. Use a recurrent neural network (RNN) to pre-process text data such as articles or comments. RNN is a neural network structure that specializes in processing sequence data. RNN has a "memory" function and can use previous information to affect subsequent calculations. It is very effective in processing time series data and natural language processing tasks. This step includes the following steps:

[0126] S221, text preprocessing, including the following steps:

[0127] S221a, Text cleaning: remove irrelevant content such as punctuation, numbers, special characters, etc. in the text.

[0128] S221b, word segmentation: the text is segmented into words or characters to obtain a text sequence. The present invention uses the word segmentation tool jieba, which is a widely used Chinese word segmentation library. For example, the word segmentation result of the sentence "I love learning" is ["I", "love", "learning"].

[0129] S221c, create a vocabulary: The technician creates a vocabulary containing all words related to used car evaluation and prediction, which is recorded as the used car evaluation and prediction vocabulary, and assigns a unique index to each word therein. For example, for the three words in the word segmentation result in step S221b, the indexes are assigned in sequence: 1, 2, 3, and the corresponding vocabulary is {"I": 1, "love": 2, "learning": 3}.

[0130] S221d, sequence encoding: Based on the index, the text sequence after word segmentation is converted into an index sequence accordingly. For example, ["I", "love", "learning"] can be encoded into an index sequence [1, 2, 3].

[0131] S221e, padding or truncation processing is performed on all index sequences so that all sequences have the same length. In the present invention, padding is performed with 0.

[0132] S222. Build an RNN model. Use a deep learning framework (such as TensorFlow, pytorch) to build an RNN model, including the following steps:

[0133] S222a, set up the embedding layer. The main function of the embedding layer is to convert discrete vocabulary indices into continuous dense vector representations, which capture the semantic information of the vocabulary in a high-dimensional space. Setting up and applying the embedding layer includes the input, operations performed, and output of the embedding layer as follows:

[0134] Embedding layer input: a sequence of indices, where each index represents a word.

[0135] The embedding layer performs operations: lookup the embedding matrix and convert the index into the corresponding embedding vector in the embedding matrix. The embedding matrix is ​​a predefined matrix that contains the vector information of all words in the vocabulary and is used to convert the word from the index form to its corresponding embedding vector.

[0136] Embedding layer output: a sequence of embedding vectors, where each embedding vector represents a high-dimensional semantic representation of a word.

[0137] For example, for a sentence "I love learning", after vocabulary encoding, the index sequence [1, 2, 3] is obtained, and the embedding layer converts this sequence into three corresponding vectors.

[0138] S222b, set up the RNN layer. The RNN layer is the core of the model and is used to process sequence data. Setting up and using the RNN layer includes the RNN layer input, the operations performed, and the output as follows:

[0139] RNN layer input: sequence of embedding vectors.

[0140] The RNN layer performs operations: At each time step, the RNN layer accepts the current input and the hidden state of the previous time step, and calculates a new hidden state. The hidden state is the internal state of the RNN layer used to store and transfer information when processing the input sequence. Among them, in the RNN, the initial value of the hidden state is set to an all-zero vector. This initial value is used as the starting point of the RNN network at the beginning of the first time step of the hidden state sequence. Definition of the initial hidden state: Assuming that the hidden state dimension of the RNN network is h, if a batch of data is processed, the batch size is b, then the shape of the initial hidden state is (b,h), b and h are integers greater than 0. The initial hidden state is initialized to an all-zero vector, that is, each element is 0. Specifically, the initial hidden state is a vector consisting of h zeros.

[0141] RNN layer output: hidden state sequence, each hidden state contains information from the beginning of the hidden state sequence to the current time step.

[0142] S222c, set the activation function. The activation function is mainly used to introduce nonlinear transformations so that the model can capture complex patterns. The activation function in this step can use the aforementioned ReLU (Rectified Linear Unit): negative values ​​in the input are set to 0, and positive values ​​remain unchanged. You can also use the Sigmoid function to compress the input between 0 and 1, or use the Softmax function to convert the input vector into a probability distribution.

[0143] S223, using the RNN model constructed in step S222 to process the data processed in step S221, a continuous text data segment in the multimodal data can obtain a feature vector of fixed length, which is recorded as the second feature vector, and the vector is usually 256-dimensional. When there are n2 (n2 is an integer greater than 1) continuous text data segments in the multimodal data, the multimodal data is processed in step S221 and input into the RNN model to obtain n2 second feature vectors, and these n2 second feature vectors are added to obtain a total 256-dimensional feature vector, and this total 256-dimensional feature vector is recorded as the text feature vector.

[0144] S23, use the linear regression model to pre-process the numerical data. Linear regression aims to make predictions by establishing a linear relationship between the target variable and the feature variable. This step specifically includes the following steps:

[0145] S231, numerical data preprocessing, including the following steps:

[0146] S231a. Use data analysis tools such as pandas’ isnull function to identify missing values ​​in numerical data. For example, data.isnull().sum() can show the number of missing values ​​in each column.

[0147] S231b. Use mean, median, mode and other methods to fill in missing data.

[0148] S231c, Identify and handle outliers in the data to reduce their impact on the model. Use the IQR method (see above for a detailed explanation of IQR) to calculate the upper and lower bounds of outliers and remove data outside these bounds.

[0149] S232. Build a linear regression model, including using the LinearRegression class to create a linear regression model instance.

[0150] S233, the linear regression model constructed in step S232 is used to process the data processed in step S231, and a numerical data in the multimodal data can obtain a fixed-length feature vector, which is recorded as the third feature vector, and the vector is usually 64-dimensional. When there are n3 (n3 is an integer greater than 1) numerical data in the multimodal data, the multimodal data is input into the linear regression model after being processed in step S231 to obtain n3 third feature vectors, and these n3 third feature vectors are added to obtain a total 64-dimensional feature vector, and this total 64-dimensional feature vector is recorded as a numerical feature vector.

[0151] S3. Perform modal fusion processing based on step S2 to combine information from different sensory data (such as pictures, text, and sound) so that the computer can more comprehensively understand and process used car related data, namely multimodal data.

[0152] Specifically, modality fusion includes the following steps:

[0153] S31, data integration: The image feature vector, text feature vector and numerical feature vector are concatenated into a comprehensive feature vector. If the image feature vector is 4096-dimensional, the text feature vector is 256-dimensional, and the numerical feature vector is 64-dimensional, then the comprehensive feature vector is 4416-dimensional.

[0154] S32. Set up a fully connected neural network (DNN) model. Including:

[0155] An input layer is set, whose size is the same as the length of the comprehensive feature vector, for receiving input data such as the feature vector.

[0156] A hidden layer is set, at least one hidden layer is added after the input layer, each hidden layer has, for example, 512 neurons, and a ReLU activation function is used. Each neuron receives input data and performs a linear transformation on it through weights and biases.

[0157] Set the output layer. Depending on whether the task type is classification or regression, the number of neurons and activation function of the output layer will be different: for binary classification tasks (in the field of used cars, binary classification tasks are used to make simple, binary judgment decisions on vehicles, which can be used for accident vehicle identification, warranty status inspection, etc. These tasks have one thing in common: they involve classifying vehicles into two mutually exclusive categories, such as "qualified / unqualified", "present / not present", etc. Binary classification models are particularly suitable for these tasks), the output layer may have 1 neuron and use the Sigmoid activation function. For multi-classification tasks, the number of neurons in the output layer is equal to the number of categories, each neuron corresponds to a category one by one, and the output layer uses the Softmax activation function. The Softmax function converts the output value of each neuron into a probability between 0 and 1 (that is, the probability of the corresponding task being achieved), and ensures that the sum of the probabilities of all outputs is 1.

[0158] In used car related tasks, the output results of the output layer depend on the specific business needs and are divided into the following three categories:

[0159] S32a. Predicting used car prices (regression task): If the task is to predict the selling price of a used car, the output layer will have a neuron, which uses a linear activation function such as the Sigmoid activation function to output a continuous value, which represents the predicted used car price.

[0160] S32b. Predicting the category of used cars (classification task): If the task is to classify the types of used cars (such as sedans, SUVs, trucks, etc.), the output layer will be provided with multiple (more than 1) neurons (each category corresponds to a neuron one-to-one), and each neuron uses the linear activation function to output a probability distribution, in which the probability represents the probability that the input used car belongs to a different type.

[0161] S32c, used car quality classification: If the task is to evaluate the quality of a used car, and the quality is divided into three quality levels: "good", "medium", and "poor", then the output layer is provided with three neurons, and each neuron also uses a linear activation function to output three probability values ​​respectively, indicating the probability of a used car belonging to each quality level.

[0162] The present invention achieves comprehensive evaluation and accurate prediction of used cars by integrating images, text and numerical data. Image data is extracted through convolutional neural networks, text data is processed through natural language processing models, and numerical data is directly used after being standardized. These features are aggregated into a unified feature vector by splicing or weighted fusion, and then input into a comprehensive model (such as a deep neural network or a Transformer model) for application. The present invention not only improves the accuracy of the assessment of the market value of used cars, but also provides personalized price predictions and quality analysis to help users make more informed purchasing decisions.

[0163] The present invention designs a series of efficient multimodal algorithm models. These models can not only effectively process complex multimodal data, but also improve the operating efficiency of the model while maintaining high accuracy, so that the present invention performs well in processing complex data and greatly improves the performance of evaluation and prediction. Specifically, these models can comprehensively process multiple data types such as images, texts and numbers, and can deeply explore the deep features in the data through innovative algorithms and optimized model structures. They not only perform well in accuracy, but also greatly improve the operating efficiency, significantly reducing the computing overhead and processing time. This high efficiency enables the model to perform well in real-time data processing and large-scale data sets; the model of the present invention has a high degree of adaptability and can flexibly respond to various complex practical application scenarios, whether in the evaluation of the value of the used car market, personalized price prediction or quality analysis, it can provide accurate and useful results. Through precise feature extraction and intelligent fusion, these models significantly improve the overall performance of evaluation and prediction, helping users make more informed decisions. In general, the present invention shows excellent performance in processing complex multimodal data and improves the ability of comprehensive evaluation and prediction.

[0164] In addition, the present invention reduces manual intervention and improves work efficiency and consistency.

[0165] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A used car evaluation and prediction method based on a multimodal algorithm model. It is characterized in that Includes steps: S1, multimodal data collection and preprocessing; S2, using a multimodal algorithm model to process the data processed in step S1; S3. Perform modal fusion processing based on step S2.

2. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 1, It is characterized in that The step S1 comprises: For text data, remove punctuation and unify capitalization to clean and standardize the text content; For numerical data, the missing values ​​are filled with the mean, median or mode of the values ​​other than the missing values; For image data, i.e. pixel values, remove noise, adjust brightness and size, and improve image quality. Here, filters are used to smooth the image and remove noise; For the text data, numerical data and image data, data deduplication is performed: duplicate samples in the data set are deleted to ensure that each data has no duplicate values.

3. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 2, It is characterized in that The filter is a Gaussian filter or a median filter.

4. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 2 or 3, It is characterized in that The step S1 further comprises the steps of: S1a. The outliers of the image data are processed as follows: the outliers are detected using the interquartile range method. The quartile is a statistic that divides a data set into four equal parts. Specifically, the interquartile range method includes: Q1 is the first quartile, representing the 25th percentile of the data set consisting of the image data, that is, 25% of the data points of the data set are below this value; Q3 is the third quartile, representing the 75th percentile of the data set consisting of the image data, that is, 75% of the data points of the data set are below this value; Let IQR = Q3-Q1, where IQR is the difference between Q3 and Q1, representing the distribution range of the middle 50% of the data set. Then there is a lower bound Lm = Q1-1.5×IQR, and an upper bound Hm = Q3+1.5×IQR. When the value of a data point in the data set is lower than the lower bound Lm, or higher than the upper bound Hm, the value of the data point is an outlier and the outlier is deleted.

5. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 4, It is characterized in that In step S2, The convolutional neural network is used to pre-process the image data, including the following steps: S211, image preprocessing, including the following steps: Adjust the image size: adjust all images to the same size to ensure consistent input format. Specifically, the pixel values ​​of the images are scaled from [0,225] to [0,1], i.e., pixel normalization. The formula is: Np = pv / 255. In the above formula, pv is the pixel value in the picture, and Np is the normalized pixel value; Data enhancement: Randomly rotate, crop, and flip the images. S212, constructing a convolutional neural network model, comprising the following steps: Set the convolution operation, the formula is as follows: The specific pixel coordinate point of the input image is recorded as the initial pixel position, and the specific pixel coordinate point in the convolution kernel is recorded as the convolution kernel pixel position. In the above formula, F(i,j) is the output value at the initial pixel position (i,j) after the convolution operation; I(i+m,j+n) is the pixel value of the input image at the initial pixel position (i+m,j+n); K(m,n) is the weight of the convolution kernel at position (convolution kernel pixel) m,n); M and N are the height and width of the convolution kernel; the scale factor α is used to adjust the output amplitude of the convolution operation, and the bias term β is used to adjust the reference value of the convolution operation output, where i, j, m, n are integers, M and N are positive integers, Set the activation function: In the above formula, α1 is a smoothing parameter, which depends on the specific working conditions. Set the pooling operation, use a fixed-size pooling window to perform the pooling operation on the feature pool, select the maximum value in each window, that is, the maximum pooling; in order to keep the size of the feature map the same as the size of the input image, add additional pixels at the boundary of the input image and fill the pixels with 0, Classification or regression, in this step, the multi-dimensional feature map after convolution and pooling is first flattened into a one-dimensional vector. The flattening operation converts the two-dimensional data into one-dimensional data so that it can be input into the fully connected layer. The fully connected layer generates new data by multiplying each input data point with a weight and adding a bias value, thereby converting the feature map data into the final result format. The output of the model, if the simulation calculation target is a classification task, use the softmax function to convert the results of the fully connected layer into the probability of each category, and output the category with the highest probability. If the simulation calculation target is a regression task, directly output a continuous numerical result to represent the specific value of the prediction. The convolutional neural network model constructed in step S212 processes the data processed in step S211, and a feature vector of fixed length is extracted from the last layer, i.e., the pooling layer, of the convolutional neural network model for the data of one of the pictures, and recorded as the first feature vector. When there are n1 pieces of data of the pictures in the multimodal data, n1 is an integer greater than 1. The multimodal data is input into the convolutional neural network model after being processed in step S211 to obtain n1 first feature vectors, and different weights are assigned to the n1 first feature vectors, and then the sum is obtained to obtain a total feature vector, which is recorded as the picture feature vector, wherein the weight is set based on the importance of the image features in the final task.

6. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 5, It is characterized in that In step S2, The recurrent neural network is used to pre-process the text data, including the following steps: S221, text preprocessing, including the following steps: Text cleaning: removing punctuation marks, numbers, and special characters from the text; Word segmentation: segment the text into words or characters to obtain a text sequence; Create vocabulary: Create a used car evaluation and prediction vocabulary and assign a unique index to each word in it; Sequence encoding: based on the index, convert the text sequence into an index sequence accordingly; padding or truncating all the index sequences so that all the index sequences have the same length, S222, constructing a recurrent neural network model, comprising the following steps: Set up an embedding layer: The input of the embedding layer is an index sequence, where each index represents a word; The operations performed by the embedding layer are: searching the embedding matrix and converting the index into a corresponding embedding vector; the output of the embedding layer is an embedding vector sequence, wherein each embedding vector represents a high-dimensional semantic representation of the word; A recurrent neural network layer is set: the input of the recurrent neural network layer is the embedding vector sequence; the operation performed by the recurrent neural network layer is: at each time step, the recurrent neural network layer accepts the current input and the hidden state of the previous time step, and calculates a new hidden state; the output of the recurrent neural network layer is a hidden state sequence, wherein each hidden state contains information from the beginning of the hidden state sequence to the current time step; Set the activation function: Same as formula (1), S223. Process the data obtained by processing step S221 with the recurrent neural network model, then a continuous text data in the multimodal data obtains a feature vector of fixed length, which is recorded as the second feature vector. When there are n2 continuous text data in the multimodal data, n2 is an integer greater than 1. The multimodal data is input into the recurrent neural network model after being processed in step S221 to obtain n2 second feature vectors. The n2 second feature vectors are added together to obtain a total feature vector, which is recorded as the text feature vector.

7. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 6, It is characterized in that In step S2, The linear regression model is used to pre-process the numerical data, including the following steps: S231, numerical data preprocessing, including the following steps: identifying missing values ​​of the numerical data; Fill in missing data; identifying and processing outliers in said numerical data, S232, build a linear regression model, including using the LinearRegression class to create a linear regression model instance, S233. Process the data obtained by processing in step S231 with the linear regression model constructed in step S232, and obtain a eigenvector of fixed length for one numerical data in the multimodal data, which is recorded as the third eigenvector. When there are n3 numerical data in the multimodal data, n3 is an integer greater than 1. The multimodal data processed in step S231 is input into the linear regression model to obtain n3 third eigenvectors. The n3 third eigenvectors are added together to obtain a total eigenvector, which is recorded as the numerical eigenvector.

8. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 7, It is characterized in that In the step S231, the interquartile range method is used to calculate the upper and lower limits of the abnormal values ​​in the numerical data, that is, after replacing the image data in the step S1a with the numerical data, the step S1a is executed again, and then the numerical data exceeding the upper and lower limits are removed.

9. The second-hand car evaluation and prediction method based on the multimodal algorithm model according to claim 8, It is characterized in that The step S3 comprises the following steps: Data integration: splicing the image feature vector, text feature vector and numerical feature vector into a comprehensive feature vector; Setting up a fully connected neural network model involves the following steps: Setting an input layer, whose size is the same as the length of the comprehensive feature vector; Setting a hidden layer, adding at least one hidden layer after the input layer, after the neurons in each hidden layer receive the input data, they perform a linear transformation on the data through weights and biases; Set the output layer. In the used car related tasks, the output results of the output layer depend on the specific business needs and are divided into the following three categories: Predicting the price of a used car: If the task is to predict the selling price of a used car, the output layer is provided with a neuron, using a linear activation function to output a continuous value, which represents the predicted price of the used car; Predicting the category of a used car: If the task is to classify the type of a used car, the output layer is provided with a plurality of neurons, each type corresponds to one of the plurality of neurons, and an activation function is used to output a probability distribution, wherein the probability represents the probability that the input used car belongs to a different type; Used car quality classification: If the task is to evaluate the quality of a used car, and the quality is divided into three quality levels: good, medium, and poor, then the output layer is provided with three neurons, which use activation functions to output three probability values, indicating the probability of a used car belonging to each quality level.

Images