Garment fashion trend prediction method and system based on images and texts

Abstract

The invention relates to the technical field of clothing fashion trend prediction, particularly provides a clothing fashion trend prediction method and system based on images and texts, and aims to solve the problem that the performance and prediction accuracy of a prediction model are poor due to the fact that the prediction of the clothing fashion trend depends on single data in the prior art. The method comprises the following steps: inputting a garment image into a trained YOLOv5 model to extract a garment feature vector, performing K-means clustering analysis to obtain a feature vector of each cluster, and analyzing a garment style or color represented by each cluster; inputting the clothing text keywords into the trained LSTM model to obtain probability value classification of the keywords; and predicting the fashion trend of the future clothing by combining the probability value classification of the clothing style or color and keyword represented by each cluster. According to the invention, the clothing image and the text data are simultaneously used to predict the clothing fashion trend, and the prediction accuracy and the performance of the clothing fashion trend prediction system can be improved.

Description

Technical Field

[0001] The invention belongs to the technical field of clothing fashion trend prediction, and in particular relates to a clothing fashion trend prediction method and system based on images and texts. Background Art

[0002] With the high-quality development of the economy and consumption upgrades, fashion has become an important factor affecting people's lifestyles, and has occupied an important position in most consumer areas, becoming one of the main battlefields for urban competition. In order to seize market hotspots and lead the high-quality development of the clothing industry, it is urgent to break through the technical bottleneck of clothing fashion trend forecasting.

[0003] In the prior art, the prediction of clothing fashion trends relies on a single data, for example, relying solely on clothing image data to predict clothing fashion trends, or relying solely on text data to predict clothing fashion trends. However, relying solely on clothing image data to predict clothing fashion trends has shortcomings such as lack of contextual information, insufficient dynamic feature capture, and high processing complexity, while relying solely on clothing text data to predict clothing fashion trends has problems such as lack of visual features, ambiguity and noise problems, cultural and language differences, etc. These two prediction methods both reduce the model prediction accuracy and prediction performance. Summary of the invention

[0004] The present invention provides a method and system for predicting clothing fashion trends based on images and texts, aiming to solve the problem that the prediction of clothing fashion trends in the prior art relies on single data, resulting in poor performance and prediction accuracy of the prediction model.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides a method for predicting clothing fashion trends based on images and texts, comprising the following steps:

[0007] S100, collect the latest clothing images and clothing text keywords through web crawlers;

[0008] S200, inputting the clothing image into the trained YOLOv5 model; the YOLOv5 model extracts the clothing feature vector of the clothing image; performing K-means clustering analysis based on the clothing feature vector to obtain the feature vector of each cluster, and analyzing the clothing style or color represented by each cluster;

[0009] S300, inputting the clothing text keywords into the trained LSTM model, and the LSTM model outputs probability value classification of keywords according to the clothing text keywords;

[0010] S400, combining the clothing styles or colors represented by each cluster in step S200 and the probability value classification of the keywords output by the LSTM model in step S300, predicting the fashion trends of future clothing.

[0011] Further solution: in the step S200, the clothing feature vector includes clothing style dimension, color dimension, and material dimension;

[0012] The clothing feature vector of each clothing image extracted by the YOLOv5 model can be expressed as:

[0013] X={x 1 , x 2 , x 3 , ..., x n};

[0014] Wherein, the X represents a clothing feature vector of the clothing image; the {x 1 , x 2 , x 3 , ..., x n} indicates that each of the clothing feature vectors includes n dimensions.

[0015] Based on the above scheme, performing clothing feature extraction on the clothing image can obtain an effective feature representation of the clothing image, reduce the complexity of calculation, and thus be used for K-means clustering analysis.

[0016] Further solution: In step S200, the K-means cluster analysis includes:

[0017] Set different K values ​​and calculate the sum of squared errors within the cluster for different K values. The calculation formula used is: Wherein, SSE(K) represents the intra-cluster error sum of squares for different K values; K represents the number of clusters; C i represents the i-th cluster; the μ i represents the centroid of the ith cluster; the x represents C i The clothing feature vector of

[0018] Draw an SSE curve according to the SSE(K), and select the K value located at the elbow of the SSE curve as the optimal K value;

[0019] After determining the optimal K value, select K initial centroids, assign each clothing image input into the YOLOv5 model to the cluster where the nearest centroid is located; update each cluster C i Definition of , calculate the new center of mass;

[0020] Using the calculated new centroid as the initial centroid, each clothing image is reassigned to the cluster where the nearest centroid is located; update each cluster C i , calculate the new centroid again; until the calculated new centroid does not change, or until the set number of iterations is reached; obtain the feature vector of each cluster;

[0021] The formula used to calculate the new center of mass is:

[0022] Based on the above scheme, by selecting the optimal K value and continuously updating the centroid, the accuracy and quality of clustering analysis can be improved, so that more accurate fashion trend prediction can be performed through the clothing images.

[0023] A further solution: before obtaining the trained YOLOv5 model in step S200 and before obtaining the trained LSTM model in step S300, historical clothing image data and text data are first collected by crawlers;

[0024] The clothing image data and text data are preprocessed respectively to obtain image samples and text samples; the YOLOv5 model is trained using the image samples to obtain a trained YOLOv5 model; and the LSTM model is trained using the text samples to obtain a trained LSTM model.

[0025] Further solution: the content of preprocessing the clothing image data includes:

[0026] The clothing pictures in the clothing picture data are marked according to the corresponding styles and colors; each clothing style and color is marked with a category and a position using a minimum circumscribed rectangular frame;

[0027] The marked clothing images are randomly rotated, randomly scaled, randomly cropped, histogram equalized and normalized, and clothing styles and colors are associated to obtain the image samples.

[0028] Based on the above scheme, the data quality of the clothing pictures can be improved by preprocessing the clothing pictures, laying a foundation for the training of the YOLOv5 model to obtain a YOLOv5 model with better performance.

[0029] Further solution: The content of preprocessing the text data includes:

[0030] Arrange the clothing styles, colors and materials in the text data according to the timeline to form time series data;

[0031] Normalizing the time series data to obtain text samples in a normally distributed form;

[0032] The formula used to normalize the time series data is:

[0033]

[0034] The n (t i ) represents a normalized text sample in a normally distributed form; the (t i ) represents time, i=1, 2, 3...n; s represents text data; max() represents the maximum value; min() represents the minimum value.

[0035] Based on the above scheme, processing the text data into time series data can improve the training effect and performance of the LSTM model, and enhance the LSTM model's analysis and prediction of popular trends based on text data.

[0036] A further solution is to build a YOLOv5 model before using the image sample to train the YOLOv5 model;

[0037] The steps to build a YOLOv5 model include:

[0038] Step 1: Setting performance indicators in the YOLOv5 model, and constructing a feature extraction unit in the YOLOv5 model according to the performance indicators; wherein the performance indicators include accuracy, recall rate, average precision, floating-point operation amount, and parameter amount;

[0039] Step 2: adding a feature fusion unit after the constructed feature extraction unit;

[0040] Step 3: Select the attention mechanism unit suitable for the image inspection task through the heat map test method, and add the selected attention mechanism unit after the fusion unit to complete the construction of the YOLOv5 model;

[0041] The learning rate and optimizer parameters are set in the constructed YOLOv5 model, and the constructed YOLOv5 model is trained using the image sample to obtain the YOLOv5 model trained in step S200.

[0042] Based on the above scheme, units with better performance are selected to jointly construct the YOLOv5 model, which also improves the overall performance of the YOLOv5 model, thereby improving the YOLOv5 model's ability to predict fashion trends based on clothing images.

[0043] Further solution: The steps of training the LSTM model using the text sample include:

[0044] Step 1: Generate a list of hyperparameter combinations using a grid search method; wherein the hyperparameters include lag size, number of hidden layers, number of neurons, dropout rate, learning rate, batch size, and epoch size;

[0045] Step 2: Convert the time series data in the text sample into supervised learning, and convert the time series data into a set of input format and output format according to the lag size; wherein the input format is represented by S input =[S 1 , S 2 , ..., S N-L ], the output format is expressed as S output =[O 1 , O 2 , ..., O N-L ], where represents the hysteresis size, and N represents the number of input or output data points;

[0046] Step 3: construct a layer of LSTM neural network according to each set of hyperparameter combinations in the hyperparameter combination list in step 1 to obtain a multi-layer LSTM neural network;

[0047] Step 4: Use the time series data in step 2 to train the multi-layer LSTM neural network, evaluate each layer of the LSTM neural network in the multi-layer LSTM neural network, and select the optimal layer of the LSTM neural network to obtain the LSTM model trained in step S300.

[0048] Based on the above solution, a list of hyperparameter combinations is generated through a grid search method, which solves the problem of manual parameter adjustment in the multi-layer LSTM neural network and can improve the stability and robustness of the multi-layer LSTM neural network.

[0049] Further solution: in step 4, each layer of the LSTM neural network is evaluated by the root mean square error; and the layer of the LSTM neural network with the smallest root mean square error is selected as the optimal layer of the LSTM neural network;

[0050] The formula for the root mean square error is:

[0051] Among them, the y t represents the actual value of the time series data at time t; represents the predicted value of the time series data at time t; and n represents the time series data.

[0052] In a second aspect, the present invention provides a clothing fashion trend prediction system based on images and texts, which is used to execute any clothing fashion trend prediction method based on images and texts as described in the first aspect; the clothing fashion trend prediction system includes a data acquisition module and a fashion trend prediction module;

[0053] The data collection module collects the latest clothing images and clothing text keywords through a web crawler; and transmits the clothing images and clothing text keywords to the fashion trend prediction module;

[0054] The popular trend prediction module includes a trained YOLOv5 model and a trained LSTM model;

[0055] The trained YOLOv5 model receives the clothing image and extracts the clothing feature vector of the clothing image; performs K-means clustering analysis based on the clothing feature vector to obtain the feature vector of each cluster, and analyzes the clothing style or color represented by each cluster;

[0056] The trained LSTM model receives the clothing text keywords and outputs probability value classification of the keywords according to the clothing text keywords;

[0057] The fashion trend prediction module combines the clothing styles or colors represented by each cluster in the trained YOLOv5 model and the probability value classification of the keywords output by the trained LSTM model to predict the future fashion trends of clothing.

[0058] The beneficial effects of the present invention are:

[0059] 1. The present invention uses two modal data as the basis for fashion trend prediction. The trained YOLOv5 model predicts fashion trends based on the clothing images, and the trained LSTM model predicts fashion trends based on clothing text keywords. The two are combined to predict fashion trends with higher accuracy, which can effectively solve the problems of insufficient dynamic feature capture, high processing complexity, lack of visual features, ambiguity and noise generated by single data prediction in the prior art.

[0060] 2. The YOLOv5 model is used in the present invention to process clothing images, which can solve the problems of slow detection speed, large amount of calculation, and cumbersome training process of the prediction model in the prior art, and further improves the performance and accuracy of the fashion trend prediction system in the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 The present invention is a flowchart of a method for predicting clothing fashion trends based on images and texts.

[0063] Figure 2 It is a structural diagram of the LSTM model in the present invention;

[0064] Figure 3 It is a structural schematic diagram of a clothing fashion trend prediction system based on images and texts in the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work belong to the protection scope of the present invention.

[0066] Embodiment 1:

[0067] like Figure 1 As shown, this embodiment provides a method for predicting clothing fashion trends based on images and texts, comprising the following steps:

[0068] S100, collect the latest clothing images and clothing text keywords through web crawlers;

[0069] S200, inputting the clothing image into the trained YOLOv5 model; the YOLOv5 model extracts the clothing feature vector of the clothing image; performing K-means clustering analysis based on the clothing feature vector to obtain the feature vector of each cluster, and analyzing the clothing style or color represented by each cluster;

[0070] S300, inputting the clothing text keywords into the trained LSTM model, and the LSTM model outputs probability value classification of keywords according to the clothing text keywords;

[0071] S400, combining the clothing styles or colors represented by each cluster in step S200 and the probability value classification of the keywords output by the LSTM model in step S300, predicting the fashion trends of future clothing.

[0072] It should be noted that the order of step S200 and step S300 can be swapped.

[0073] A more specific example of step S100 in the above scheme is: in step S100, the latest clothing images and clothing text keywords are from a period relatively close to the present time, for example, one month ago, half a year ago, nearly one year ago, nearly two years ago, etc. The sources of the latest clothing images and clothing text keywords can be shopping platforms (for example, Amazon, Alibaba, Taobao, Tmall, JD.com, NetEase Kaola, Anta, etc.), social platforms (for example, Facebook, Instagram, Twitter, WeChat, Weibo, etc.), fashion media magazines (for example, Vogue, Elle, GQ, InStyle, WMagazine, Grazie, etc.) or fashion weeks (for example, New York Fashion Week, London Fashion Week, Milan Fashion Week, Paris Fashion Week, Shanghai Fashion Week, Tokyo Fashion Week, Seoul Fashion Week, Los Angeles Fashion Week, Berlin Fashion Week, Sydney Fashion Week, etc.), etc.

[0074] A more specific example of step S200 in the above scheme is as follows: in step S200, the clothing feature vector includes clothing style dimension (e.g., T-shirt, skirt, windbreaker, etc.), color dimension (e.g., blue, yellow, red, etc.), and material dimension (e.g., cotton, silk, etc.);

[0075] The clothing feature vector of each clothing image extracted by the YOLOv5 model can be expressed as:

[0076] X={x 1 , x 2 , x 3 , ..., x n};

[0077] Wherein, the X represents a clothing feature vector of the clothing image; the {x 1 , x 2 , x 3 , ..., x n} indicates that each of the clothing feature vectors includes n dimensions.

[0078] A further example of the above solution is: in step S200, the K-means cluster analysis includes:

[0079] Set different K values ​​and calculate the sum of squared errors within the cluster for different K values. The calculation formula used is: Wherein, SSE(K) represents the intra-cluster error sum of squares for different K values; K represents the number of clusters; C irepresents the i-th cluster; the μ i represents the centroid of the ith cluster; the x represents C i The clothing feature vector of

[0080] Draw an SSE curve according to the SSE(K), and select the K value located at the elbow of the SSE curve as the optimal K value;

[0081] After determining the optimal K value, select K initial centroids, assign each clothing image input into the YOLOv5 model to the cluster where the nearest centroid is located; update each cluster C i Definition of , calculate the new center of mass;

[0082] Using the calculated new centroid as the initial centroid, each clothing image is reassigned to the cluster where the nearest centroid is located; update each cluster C i , calculate the new centroid again; until the calculated new centroid does not change, or until the set number of iterations is reached; obtain the feature vector of each cluster;

[0083] The formula used to calculate the new center of mass is:

[0084] A more specific example of step S300 in the above solution is: Figure 2 As shown, the structure of the LSTM model is expressed as:

[0085] a(t i )=σ(w a x(t i )+w ha h(t i-1 )+b a ); Formula 1

[0086] f(t i )=σ(w f x(t i )+w hf h(t i-1 )+b f ); Formula 2

[0087] c(t i )=f(t i )×c(t i-1 )+a(t i )×tanh(w c x(t i )+w hc h(t i-1 )+b c ); Formula 3

[0088] o(t i)=σ(w o x(t i )+w ho h(t i-1 )+b o ); Formula 4

[0089] h(t i )=o(t i )×tanh(c(t i )); Formula 5

[0090] Wherein, in Formula 1, a(ti) is the input gate; σ is the sigmoid activation function; w a is the weight matrix of the current state input gate; the x(t i ) is the current state input value; the w ha is the weight matrix of the hidden state input gate; the h(t i-1 ) is the hidden state of the previous moment; a is the bias term of the input gate. In Formula 2, f(t i ) is a forget gate; the w f is the weight matrix of the current state forget gate; hf is the weight matrix of the hidden state forget gate; f is the bias term of the forget gate. In Formula 3, c(t i ) is the cell state; the c(t i-1 ) is the cell state at the previous moment; the × is point-by-point multiplication; the tanh is the hyperbolic tangent activation function; the w c is the weight matrix of candidate information; the w hc is the weight matrix of the hidden state candidate information; c is the bias term of the candidate information. In Formula 4, o(t i ) is the output gate; the w o is the weight matrix of the output gate of the current state; the w ho is the weight matrix of the hidden state output gate; o is the bias term of the output gate. In Formula 5, h(t i )The hidden state at the current moment.

[0091] It should be noted that: Figure 2 In the above, c′(t i )=tanh(w c x(t i )+w hc h(t i-1 )+b c ).

[0092] On the basis of the above scheme, before obtaining the trained YOLOv5 model in step S200 and before obtaining the trained LSTM model in step S300, historical clothing image data and text data are first collected by crawlers;

[0093] The clothing image data and text data are preprocessed respectively to obtain image samples and text samples; the YOLOv5 model is trained using the image samples to obtain a trained YOLOv5 model; and the LSTM model is trained using the text samples to obtain a trained LSTM model.

[0094] It should be noted that: when collecting historical clothing image data and text data, in addition to using crawlers, you can also collect them manually or through methods provided by merchants; the sources of collecting historical clothing image data and text data can be social media and shopping platforms, etc.

[0095] In order to enable the clothing image data and text data to be recognized and processed, the clothing image data and text data need to be preprocessed;

[0096] Preprocessing 1: The content of preprocessing the clothing image data (image resolution is 1920×1080) includes:

[0097] The clothing pictures in the clothing picture data are marked according to the corresponding styles and colors; each clothing style and color is marked with a category and a position using a minimum circumscribed rectangular frame;

[0098] The marked clothing images are randomly rotated, randomly scaled, randomly cropped, histogram equalized and normalized, and clothing styles and colors are associated to obtain the image samples.

[0099] Specifically, when each clothing style and color is marked with a category and a position, the minimum bounding rectangle should contain the clothing and be marked as (class, xmin, ymin, xmax, ymax). Among them, class represents clothing style and color, xmin and ymin represent the horizontal coordinate and vertical coordinate of the upper left corner of the minimum bounding rectangle, and xmax and ymax represent the horizontal coordinate and vertical coordinate of the lower right corner of the minimum bounding rectangle.

[0100] Preprocessing 2: The content of preprocessing the text data includes:

[0101] Arrange the clothing styles, colors and materials in the text data according to the timeline to form time series data;

[0102] Normalizing the time series data to obtain text samples in a normally distributed form;

[0103] The formula used to normalize the time series data is:

[0104]

[0105] The n (t i ) represents a normalized text sample in a normally distributed form; the (t i ) represents time, i=1, 2, 3...n; s represents text data; max() represents the maximum value; min() represents the minimum value.

[0106] Specifically, when the clothing styles, colors and materials in the text data are arranged according to a timeline to form time series data, if the time series data contains noise values ​​and missing values, it is necessary to smooth the noise values ​​(for example, use the moving average method to calculate the average value in the window to smooth the data) and replace the missing values ​​(for example, use the linear interpolation method to infer the missing values ​​based on the existing data).

[0107] The following is a specific example of training the YOLOv5 model:

[0108] Before using the image sample to train the YOLOv5 model, a YOLOv5 model is first constructed; the steps of constructing the YOLOv5 model include:

[0109] Step 1: Set the performance indicators in the YOLOv5 model, and build the feature extraction unit in the YOLOv5 model according to the performance indicators; wherein the performance indicators include precision, recall, mean average precision (mAP), floating point operations (FLOPs) and parameters;

[0110] Step 2: adding a feature fusion unit after the constructed feature extraction unit;

[0111] Step 3: Select the attention mechanism unit suitable for the image inspection task through the heat map test method, and add the selected attention mechanism unit after the fusion unit to complete the construction of the YOLOv5 model;

[0112] The learning rate and optimizer parameters are set in the constructed YOLOv5 model, and the constructed YOLOv5 model is trained using the image sample to obtain the YOLOv5 model trained in step S200.

[0113] A more specific example of the above scheme is: in the first step, the accuracy Wherein the TP (true positive) represents the number of correctly identified image samples; the FP (false positive) represents the number of incorrectly identified image samples;

[0114] The recall rate The FN (false negative) represents the number of image samples that were not recognized;

[0115] The average precision The N represents the number of image samples; the AP represents the average precision of one image sample; the P(R) represents the precision under different recall rates; and the dR represents the integral;

[0116] The floating point operation amount FLOPs=W×H×K×K×C in ×C out ; W represents the width of the input image sample feature map; H represents the height of the input image sample feature map; K represents the size of the convolution kernel; C in Indicates the number of input channels; the C out Indicates the number of output channels;

[0117] The parameters are parameters = C in ×C out ×K×K.

[0118] In the second step, since there are multiple attention mechanisms (such as self-attention, channel attention, spatial attention, etc.), in order to select an attention mechanism suitable for image inspection tasks, it is necessary to test multiple attention mechanisms through heat map testing. The steps include:

[0119] Integrate multiple attention mechanisms into a basic model (for example: ResNet, YOLO, etc.); for each detection image in the input model, calculate the gradient of the model output and the target category, and calculate the weight of the feature map of each detection image; perform weighted summation of the feature map through the weights and process it through the ReLU activation function to obtain a heat map; superimpose the heat map with the detection image, observe and analyze the heat map generated by each attention mechanism, such as how each attention mechanism focuses on different areas of the detection image, so as to select the required attention mechanism suitable for the image inspection task.

[0120] The following is a specific example of training an LSTM model:

[0121] The steps of training the LSTM model using the text sample include:

[0122] Step 1: Generate a list of hyperparameter combinations using a grid search method; wherein the hyperparameters include lag size, number of hidden layers, number of neurons, dropout rate, learning rate, batch size, and epoch size;

[0123] Step 2: Convert the time series data in the text sample into supervised learning, and convert the time series data into a set of input format and output format according to the lag size; wherein the input format is represented by S input =[S 1 , S 2 , ..., S N-L ], the output format is expressed as S output =[O 1 , O 2 , ..., O N-L ], where represents the hysteresis size, and N represents the number of input or output data points;

[0124] Step 3: construct a layer of LSTM neural network according to each set of hyperparameter combinations in the hyperparameter combination list in step 1 to obtain a multi-layer LSTM neural network;

[0125] Step 4: Use the time series data in step 2 to train the multi-layer LSTM neural network, evaluate each layer of the LSTM neural network in the multi-layer LSTM neural network, and select the optimal layer of the LSTM neural network to obtain the LSTM model trained in step S300.

[0126] A more specific example of the above scheme is: in step 1, the network search method is a hyperparameter optimization method. A predefined value is set for each hyperparameter; through an exhaustive search, the predefined hyperparameter set is traversed to find the best hyperparameter combination and generate a hyperparameter combination list.

[0127] For the lag size, it has a significant impact on the performance of time series data prediction, so it is crucial to test the performance of LSTM neural network using different lag sizes. For the number of hidden layers, deep neural network architecture has better generalization ability than single-layer architecture, so it is important to study the performance of a multi-layer LSTM neural network. For the number of neurons, it helps to select the optimal number of neurons for each layer in the multi-layer LSTM neural network. For the dropout rate, applying the dropout rate in the multi-layer LSTM neural network affects the performance of the multi-layer LSTM neural network by enhancing generalization. For the learning rate, it is possible to adjust the degree of change in the weights of the multi-layer LSTM neural network. For the batch size, adjusting the batch size is another factor that determines the performance of the multi-layer LSTM neural network. For the epoch size; finding the right number of epochs is crucial to achieve a multi-layer LSTM neural network with high performance.

[0128] In the step 4, each layer of the LSTM neural network is evaluated by the root mean square error; and the layer of the LSTM neural network with the smallest root mean square error is selected as the optimal layer of the LSTM neural network;

[0129] The formula for the root mean square error is:

[0130] Wherein, y=t represents the actual value of the time series data at time t; represents the predicted value of the time series data at time t; and n represents the time series data.

[0131] Embodiment 2:

[0132] like Figure 3 As shown, this embodiment provides a clothing fashion trend prediction system based on images and texts, which is used to execute a clothing fashion trend prediction method based on images and texts described in Embodiment 1; the clothing fashion trend prediction system includes a data acquisition module and a fashion trend prediction module;

[0133] The data collection module collects the latest clothing images and clothing text keywords through a web crawler; and transmits the clothing images and clothing text keywords to the fashion trend prediction module;

[0134] The popular trend prediction module includes a trained YOLOv5 model and a trained LSTM model;

[0135] The trained YOLOv5 model receives the clothing image and extracts the clothing feature vector of the clothing image; performs K-means clustering analysis based on the clothing feature vector to obtain the feature vector of each cluster, and analyzes the clothing style or color represented by each cluster;

[0136] The trained LSTM model receives the clothing text keywords and outputs probability value classification of the keywords according to the clothing text keywords;

[0137] The fashion trend prediction module combines the clothing styles or colors represented by each cluster in the trained YOLOv5 model and the probability value classification of the keywords output by the trained LSTM model to predict the future fashion trends of clothing.

[0138] The present invention is not limited to the above-mentioned optional implementation modes. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that fall within the scope defined by the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A clothing fashion trend prediction method based on images and texts, characterized in that: The following steps are involved: S100, collect the latest clothing images and clothing text keywords through web crawlers; S200, inputting the clothing image into the trained YOLOv5 model; the YOLOv5 model extracts the clothing feature vector of the clothing image; performing K-means clustering analysis based on the clothing feature vector to obtain the feature vector of each cluster, and analyzing the clothing style or color represented by each cluster; S300, inputting the clothing text keywords into the trained LSTM model, and the LSTM model outputs probability value classification of keywords according to the clothing text keywords; S400, combining the clothing styles or colors represented by each cluster in step S200 and the probability value classification of the keywords output by the LSTM model in step S300, predicting the fashion trends of future clothing.

2. The method for predicting clothing fashion trends based on images and texts according to claim 1, characterized in that: In the step S200, the clothing feature vector includes clothing style dimension, color dimension, and material dimension; The clothing feature vector of each clothing image extracted by the YOLOv5 model can be expressed as: X={x1,x2,x3,...,x n }; Wherein, the X represents a clothing feature vector of the clothing image; the {x1, x2, x3, ..., x n } indicates that each of the clothing feature vectors includes n dimensions.

3. The method for predicting clothing fashion trends based on images and texts according to claim 2, characterized in that: In step S200, the K-means cluster analysis includes: Set different K values ​​and calculate the sum of squared errors within the cluster for different K values. The calculation formula used is: Wherein, SSE(K) represents the intra-cluster error sum of squares for different K values; K represents the number of clusters; C i represents the i-th cluster; μi represents the centroid of the i-th cluster; x represents C i The clothing feature vector of Draw an SSE curve according to the SSE(K), and select the K value located at the elbow of the SSE curve as the optimal K value; After determining the optimal K value, select K initial centroids, assign each clothing image input into the YOLOv5 model to the cluster where the nearest centroid is located; update each cluster C i Definition of , calculate the new center of mass; Using the calculated new centroid as the initial centroid, each clothing image is reassigned to the cluster where the nearest centroid is located; update each cluster C i , calculate the new centroid again; until the calculated new centroid does not change, or until the set number of iterations is reached; obtain the feature vector of each cluster; The formula used to calculate the new center of mass is:

4. The method for predicting clothing fashion trends based on images and texts according to claim 1, characterized in that: Before obtaining the trained YOLOv5 model in step S200 and before obtaining the trained LSTM model in step S300, historical clothing image data and text data are first collected by crawlers; The clothing image data and text data are preprocessed respectively to obtain image samples and text samples; the YOLOv5 model is trained using the image samples to obtain a trained YOLOv5 model; and the LSTM model is trained using the text samples to obtain a trained LSTM model.

5. The method for predicting clothing fashion trends based on images and texts according to claim 4, characterized in that: The content of preprocessing the clothing image data includes: The clothing pictures in the clothing picture data are marked according to the corresponding styles and colors; each clothing style and color is marked with a category and a position using a minimum circumscribed rectangular frame; The marked clothing images are randomly rotated, randomly scaled, randomly cropped, histogram equalized and normalized, and clothing styles and colors are associated to obtain the image samples.

6. The method for predicting clothing fashion trends based on images and texts according to claim 4, characterized in that: The content of preprocessing the text data includes: Arrange the clothing styles, colors and materials in the text data according to the timeline to form time series data; Normalizing the time series data to obtain text samples in a normally distributed form; The formula used to normalize the time series data is: The n (t i ) represents a normalized text sample in a normally distributed form; the (t i ) represents time, i=1, 2, 3...n; s represents text data; max() represents the maximum value; min() represents the minimum value.

7. The method for predicting clothing fashion trends based on images and texts according to claim 5, characterized in that: Building a YOLOv5 model before using the image sample to train the YOLOv5 model; The steps to build a YOLOv5 model include: Step 1: Setting performance indicators in the YOLOv5 model, and constructing a feature extraction unit in the YOLOv5 model according to the performance indicators; wherein the performance indicators include accuracy, recall rate, average precision, floating-point operation amount, and parameter amount; Step 2: adding a feature fusion unit after the constructed feature extraction unit; Step 3: Select the attention mechanism unit suitable for the image inspection task through the heat map test method, and add the selected attention mechanism unit after the fusion unit to complete the construction of the YOLOv5 model; The learning rate and optimizer parameters are set in the constructed YOLOv5 model, and the constructed YOLOv5 model is trained using the image sample to obtain the YOLOv5 model trained in step S200.

8. The method for predicting clothing fashion trends based on images and texts according to claim 6, characterized in that: The steps of training the LSTM model using the text sample include: Step 1: Generate a list of hyperparameter combinations using a grid search method; wherein the hyperparameters include lag size, number of hidden layers, number of neurons, dropout rate, learning rate, batch size, and epoch size; Step 2: Convert the time series data in the text sample into supervised learning, and convert the time series data into a set of input format and output format according to the lag size; wherein the input format is represented by S input =[S1, S2, ..., S N-L ], the output format is expressed as S output =[O1, O2, ..., O N-L ], where represents the hysteresis size, and N represents the number of input or output data points; Step 3: construct a layer of LSTM neural network according to each set of hyperparameter combinations in the hyperparameter combination list in step 1 to obtain a multi-layer LSTM neural network; Step 4: Use the time series data in step 2 to train the multi-layer LSTM neural network, evaluate each layer of the LSTM neural network in the multi-layer LSTM neural network, and select the optimal layer of the LSTM neural network to obtain the LSTM model trained in step S300.

9. The method for predicting clothing fashion trends based on images and texts according to claim 8, characterized in that: In the step 4, each layer of the LSTM neural network is evaluated by the root mean square error; and the layer of the LSTM neural network with the smallest root mean square error is selected as the optimal layer of the LSTM neural network; The formula for the root mean square error is: Wherein, yt represents the actual value of the time series data at time t; represents the predicted value of the time series data at time t; and n represents the time series data.

10. A clothing fashion trend prediction system based on images and texts, characterized in that: Used to execute a clothing fashion trend prediction method based on images and texts as described in any of claims 1-9; the clothing fashion trend prediction system includes a data acquisition module and a fashion trend prediction module; The data collection module collects the latest clothing images and clothing text keywords through a web crawler; and transmits the clothing images and clothing text keywords to the fashion trend prediction module; The popular trend prediction module includes a trained YOLOv5 model and a trained LSTM model; The trained YOLOv5 model receives the clothing image and extracts the clothing feature vector of the clothing image; performs K-means clustering analysis based on the clothing feature vector to obtain the feature vector of each cluster, and analyzes the clothing style or color represented by each cluster; The trained LSTM model receives the clothing text keywords and outputs probability value classification of the keywords according to the clothing text keywords; The fashion trend prediction module combines the clothing styles or colors represented by each cluster in the trained YOLOv5 model and the probability value classification of the keywords output by the trained LSTM model to predict the future fashion trends of clothing.

Images