An artificial intelligence-based personalized clothing recommendation method, system and medium
By working in tandem with a pre-built image encoder and an NLU module, and combining multiple independent feature extraction models, the clothing recommendation strategy is dynamically adjusted. This solves the problem of insufficient fine-grained attribute representation in existing technologies, achieving efficient and accurate personalized clothing recommendations and improving the user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO KUTE INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2025-07-14
- Publication Date
- 2026-05-19
AI Technical Summary
Existing clothing image similarity search relies on a global vector output by a single network, resulting in insufficient fine-grained attribute representation and an inability to dynamically adjust according to real-time user needs. Furthermore, existing clothing image retrieval and recommendation strategies are relatively fixed, making it difficult to meet diverse user needs and impacting the user experience of personalized clothing recommendations.
It employs a pre-built image encoder and an NLU module to work together to parse user-input query data, capture different dimensions of clothing features through multiple independent feature extraction models, dynamically adjust the retrieval strategy, support diverse retrieval needs, and generate accurate clothing recommendation results.
It improves the accuracy and efficiency of clothing recommendation retrieval, meets diverse user needs, enhances the user experience of personalized clothing recommendations, and achieves high quality and flexibility in clothing retrieval.
Smart Images
Figure CN120852010B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of e-commerce technology, and in particular to a method, system and medium for personalized clothing recommendation based on artificial intelligence. Background Technology
[0002] In recent years, with the booming development of e-commerce and the fashion industry, consumers' demand for clothing image retrieval is shifting from traditional fuzzy retrieval to fine-grained, personalized and interactive retrieval.
[0003] However, existing clothing image similarity searches mostly rely on global vectors output by a single network, which lacks sufficient representation of fine-grained clothing attributes and cannot dynamically adjust according to real-time user needs. Furthermore, existing clothing image retrieval and recommendation strategies are relatively fixed and singular, failing to meet diverse user requirements. These issues result in insufficient dynamic adaptability of feature representations and retrieval strategies in personalized clothing recommendations, negatively impacting the user experience when using clothing recommendation systems.
[0004] Therefore, there is an urgent need for a technical solution for personalized clothing recommendations to improve the user experience of clothing recommendation systems. Summary of the Invention
[0005] This application provides a personalized clothing recommendation method, system, and medium based on artificial intelligence, which addresses the technical problem of insufficient dynamic adaptability of feature representation and retrieval strategies in current personalized clothing recommendations, resulting in a poor user experience.
[0006] In a first aspect, embodiments of this application provide a personalized clothing recommendation method based on artificial intelligence, the method comprising:
[0007] Based on a pre-set query data parsing module, the query semantic information and attribute weight vector corresponding to the query data input by the user are determined; wherein, the pre-set query data parsing module includes at least a pre-set image encoder and a pre-set NLU module; the query semantic information includes at least query intent and query attribute parameters;
[0008] Based on the query semantic information, the attribute weight vector, and the pre-constructed attribute module set, the corresponding attribute fusion feature information is determined; the attribute module set includes feature extraction modules corresponding to multiple preset common clothing attributes.
[0009] Based on the attribute fusion feature information, the retrieval strategy from the user terminal, and the preset ANN algorithm, several candidate clothing images in the clothing image database are matched, and an image sorting sequence is generated through a preset re-sorting model; wherein, the retrieval strategy includes at least overall fusion feature retrieval, single-dimensional feature retrieval, and multi-path single-dimensional feature retrieval.
[0010] Based on the image sorting sequence, the corresponding recommended clothing images are sent to the user terminal.
[0011] In one implementation of this application, a pre-built query data parsing module is used to determine the query semantic information and attribute weight vector corresponding to the user-input query data, specifically including:
[0012] After receiving the query data input by the user, the data type of the query data is determined; the data type includes at least one of the following: text type, image type, and image and text type;
[0013] Based on the data type, the preset image encoder and / or the preset NLU module in the preset query data parsing module are invoked to determine the global feature vector corresponding to the query data;
[0014] The global feature vector is matched with a preset historical semantic data list to determine the corresponding query semantic information based on the matching result;
[0015] The global feature vector is input into a pre-trained weight prediction network model to determine the attribute weight vector based on the model output.
[0016] In one implementation of this application, the method further includes:
[0017] When the query data includes multiple reference images, one or more specified reference attributes are obtained from the user terminal; wherein, the specified reference attributes have a mapping relationship with the reference images and are used to characterize the attribute feature dimension of the attribute feature vector extracted from the corresponding reference image;
[0018] Based on the specified reference attribute and the attribute module set, the corresponding attribute feature vectors in each reference image are determined and input into the downstream attribute fusion processing task.
[0019] In one implementation of this application, each feature extraction module in the pre-built set of attribute modules corresponds to an independent common clothing attribute, and the common clothing attribute includes at least one of the following: color, material, style, and type;
[0020] Before determining the corresponding attribute fusion feature information based on the query semantic information, the attribute weight vector, and the pre-constructed set of attribute modules, the method further includes:
[0021] Acquire several feature extraction sample data corresponding to different common clothing attributes; the feature extraction sample data includes at least image-text pair association data, fine-grained specific attribute annotation data, and single-modal feature enhancement data;
[0022] According to the common clothing attributes, the feature extraction sample data is input into the feature extraction module to be trained, so as to train the image-text dual encoder, attribute text encoder and image encoder in the feature extraction module respectively, until the loss function value of the feature extraction module is less than a predetermined value, and the trained feature extraction module is obtained. Then, the feature extraction modules corresponding to each of the common clothing attributes are added to the attribute module set.
[0023] In one implementation of this application, the corresponding attribute fusion feature information is determined based on the query semantic information, the attribute weight vector, and a pre-constructed set of attribute modules, specifically including:
[0024] Based on the common clothing attributes corresponding to the query semantic information, the retrieval participation feature extraction modules in the attribute module set are determined;
[0025] The query data is input into the retrieval participation feature extraction module to determine one or more corresponding attribute feature vectors based on the feature extraction results;
[0026] Based on the weighted calculation results of the attribute weight vector and each attribute feature vector, an attribute fusion feature vector is determined, and the weighted attribute feature vector and the attribute fusion feature vector corresponding to each attribute feature vector are used as the attribute fusion feature information.
[0027] In one implementation of this application, the attribute fusion feature vector is determined based on the weighted calculation results of the attribute weight vector and each attribute feature vector, using one of the following methods:
[0028] Based on the attribute weight vector, the attribute feature vectors are linearly combined to calculate the attribute fusion feature vector V_fused = w1·V1 + w2·V2 + ... + w n ·V n , where w n V represents the attribute weight corresponding to the nth common clothing attribute. n The attribute feature vector corresponding to the nth common clothing attribute;
[0029] The attribute weight vector and each attribute feature vector are input into a pre-trained adaptive feature selection model to calculate the attribute fusion feature vector V_fused; the adaptive feature selection model includes at least one of attention mechanism and gating network.
[0030] In one implementation of this application, based on the attribute fusion feature information, the retrieval strategy from the user terminal, and a preset ANN algorithm, several candidate clothing images in the clothing image database are matched, specifically including:
[0031] Receive the retrieval strategy from the user terminal and determine the respective interest feature vectors of the corresponding interest attributes in the attribute fusion feature information;
[0032] According to the retrieval strategy, one or more of the aforementioned feature vectors are recombined, and the recombined feature vector group is matched with the feature vectors of clothing images in the clothing image database for similarity. Based on the similarity matching results, candidate clothing image feature vectors are determined, resulting in several candidate clothing images.
[0033] In one implementation of this application, an image sorting sequence is generated through a preset re-sorting model, specifically including:
[0034] The candidate clothing image feature vectors and their corresponding attention feature vectors for each candidate clothing image are masked. The candidate clothing image feature vectors, the attention feature vectors, the masked candidate clothing image feature vectors and the masked attention feature vectors are then concatenated to obtain an initial feature concatenation vector. The initial feature concatenation vector has a one-to-one correspondence with the candidate clothing images.
[0035] The initial feature concatenation vector is input into the preset reordering model to score the feature vectors of the candidate clothing images and determine the recommended score for each candidate clothing image.
[0036] According to the recommended scores in descending order, the labels of each candidate clothing image are added to the image sorting sequence in sequence.
[0037] Secondly, embodiments of this application also provide a personalized clothing recommendation system based on artificial intelligence, the system comprising:
[0038] The first determining module is used to determine the query semantic information and attribute weight vector corresponding to the query data input by the user, based on the preset query data parsing module; wherein, the preset query data parsing module includes at least a preset image encoder and a preset NLU module; the query semantic information includes at least query intent and query attribute parameters;
[0039] The second determining module is used to determine the corresponding attribute fusion feature information based on the query semantic information, the attribute weight vector, and the pre-constructed attribute module set; the attribute module set includes feature extraction modules corresponding to multiple preset common clothing attributes.
[0040] The matching generation module is used to match several candidate clothing images in the clothing image database according to the attribute fusion feature information, the retrieval strategy from the user terminal and the preset ANN algorithm, and generate an image ranking sequence through a preset re-ranking model; wherein, the retrieval strategy includes at least overall fusion feature retrieval, single-dimensional feature retrieval and multi-way single-dimensional feature retrieval.
[0041] The sending module is used to send the corresponding recommended clothing images to the user terminal according to the image sorting sequence.
[0042] Thirdly, embodiments of this application also provide a non-volatile computer storage medium storing computer-executable instructions, which are capable of executing the artificial intelligence-based personalized clothing recommendation method described above.
[0043] Compared with the prior art, the significant advantages of this application are as follows:
[0044] This application, through the aforementioned scheme, utilizes a pre-built image encoder and an NLU module to simultaneously process user-input image and text information, accurately identifying query intent and attribute parameters. It employs multiple independent feature extraction models to capture different dimensions of clothing features, dynamically meeting diverse retrieval needs, reducing irrelevant feature interference, and improving retrieval efficiency and accuracy. Furthermore, this application supports diverse retrieval strategies, covering various scenario retrieval needs, and flexibly and systematically outputting clothing recommendation results, thus enhancing user experience. It solves the problem of insufficient dynamic adaptability of feature representation and retrieval strategies in current personalized clothing recommendations, significantly improving the accuracy of personalized recommendations and user experience, and meeting the urgent needs of e-commerce and fashion industries for high-quality clothing retrieval. Attached Figure Description
[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0046] Figure 1 This is a flowchart illustrating a personalized clothing recommendation method based on artificial intelligence, as described in an embodiment of this application.
[0047] Figure 2 This is a schematic diagram of the structure of an artificial intelligence-based personalized clothing recommendation system in an embodiment of this application. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0049] Based on this, embodiments of this application provide a personalized clothing recommendation method, system, and medium based on artificial intelligence to solve the technical problem that the dynamic adaptability of feature representation and retrieval strategies in current personalized clothing recommendations is insufficient, resulting in a poor user experience.
[0050] The various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0051] This application provides an embodiment of a personalized clothing recommendation method based on artificial intelligence, such as... Figure 1 As shown, the method may include steps S101-S104:
[0052] S101, the server determines the query semantic information and attribute weight vector corresponding to the query data input by the user based on the pre-set query data parsing module.
[0053] The pre-configured query data parsing module includes at least a pre-configured image encoder and a pre-configured NLU module. Query semantic information includes at least the query intent and query attribute parameters. The query intent may include intents such as attribute replacement and multi-image joint retrieval, while the query attribute parameters may include target attributes, image reference indexes, and weight preferences.
[0054] It should be noted that the server, as the executor of the AI-based personalized clothing recommendation method, exists only as an example, and the executor is not limited to the server. This application does not make any specific limitation in this regard.
[0055] In this embodiment of the application, based on a pre-set query data parsing module, the query semantic information and attribute weight vector corresponding to the query data input by the user are determined, specifically including:
[0056] Upon receiving the user's query data, the data type is determined. The data type must include at least one of the following: text, image, or image-text. Based on the data type, the pre-set image encoder and / or pre-set NLU module in the pre-set query data parsing module are invoked to determine the global feature vector corresponding to the query data. The global feature vector is matched against a pre-set historical semantic data list to determine the corresponding query semantic information based on the matching results. The global feature vector is then input into a pre-trained weight prediction network model to determine the attribute weight vector based on the model's output.
[0057] In other words, the server can run corresponding software or platforms with AI-based personalized clothing recommendation methods, and the user terminal can run mobile or web-based software that interacts with the server. Users send query data for clothing retrieval to the server through their terminals, and the server then executes the relevant query operations. Specifically, the query data can be text input by the user, such as the text "blue T-shirt," or image data, such as a photo of a T-shirt, or image-text data, such as a photo of clothing and a text description of the content they are interested in from the photo.
[0058] Then, the server, based on the data type of the query data, will use one or more of the preset image encoder and preset NLU modules from the preset query data parsing module to generate a global feature vector for the query data. The preset NLU module includes a preset Natural Language Understanding (NLU) model from artificial intelligence, which can parse the user's natural language to determine the user's focus attributes and other necessary clothing attributes, and add them to the global feature vector. The preset image encoder can extract image attribute features and map them to a preset attribute subspace of common clothing attributes, thereby generating corresponding encoded vectors, which are then added to the global feature vector.
[0059] Subsequently, the server will match the query semantic information corresponding to the global feature vector using a pre-stored, preset historical semantic data list. This preset historical semantic data list contains the correspondence between several feature vectors and query semantic information. During matching, the server can calculate the cosine similarity between the global feature vector and each feature vector in the list, and determine the query semantic information associated with the feature vector corresponding to the maximum cosine similarity in the calculation results, which is then used as the query semantic information of the global feature vector. Simultaneously, the server also includes a weight prediction network model trained on several feature vector samples carrying attribute weight vectors. This model analyzes and processes the global feature vector to obtain the corresponding attribute weight vector. The weight prediction network model can employ a multilayer perceptron (MLP) or a network model containing a self-attention layer; this application does not specifically limit its use.
[0060] Specifically, the output of the weighted prediction network model is a fixed-dimensional vector w, whose dimension is equal to the total number of modules with predefined shared clothing attributes in the system (for example, if there are four feature extraction modules: color, material, style, and texture (including lines), then the output is a 4-dimensional vector). Each element w in the vector w... iThis represents the importance weight of the i-th attribute module. The output is typically processed using Softmax or Sigmoid activation functions to normalize the weights (e.g., sum to 1 or in the [0, 1] interval). Example: Given the input text "red wool sweater," the weight prediction network model, after analyzing its global feature vector, might output a weight distribution similar to w, where color and material have higher weights (e.g., w_color≈0.4, w_material≈0.4), while style and texture (including lines) have relatively lower weights (e.g., w_style≈0.15, w_texture≈0.05). Inputting "blue silk dress" will generate a different weight distribution.
[0061] S102, the server determines the corresponding attribute fusion feature information based on the query semantic information, attribute weight vector, and pre-built attribute module set.
[0062] The attribute module set includes feature extraction modules corresponding to multiple preset common clothing attributes.
[0063] In this embodiment of the application, each feature extraction module in the pre-built attribute module set corresponds to an independent common clothing attribute, and the common clothing attribute includes at least one of the following: color, material, style and type.
[0064] Before determining the corresponding attribute fusion feature information based on query semantic information, attribute weight vectors, and a pre-constructed set of attribute modules, the method further includes:
[0065] Obtain several feature extraction sample data corresponding to different shared clothing attributes. The feature extraction sample data includes at least image-text pair association data, fine-grained specific attribute annotation data, and single-modal feature enhancement data. According to the shared clothing attributes, input the feature extraction sample data into the feature extraction module to be trained, so as to train the image-text dual encoder, attribute text encoder, and image encoder in the feature extraction module respectively, until the loss function value of the feature extraction module is less than a predetermined value, and obtain the trained feature extraction module. Then, add the trained feature extraction modules corresponding to each shared clothing attribute to the attribute module set.
[0066] In other words, this application uses several feature extraction sample data to train the feature extraction module. Specifically, it trains the image-text dual encoder using image-text pair association data, the attribute text encoder using fine-grained specific attribute annotation data, and the image encoder using single-modal feature enhancement data, thereby training the aforementioned feature extraction module. The predetermined values mentioned above can be set by the user according to the actual usage scenario and are not specifically limited here. The fine-grained specific attribute annotation data can be understood as several fine-grained data contained in various dimensions of different common clothing attributes. For example, the fine-grained specific attribute annotation data for color includes red, yellow, pink, etc.; the fine-grained specific attribute annotation data for material includes wool, pure cotton, etc. This application does not specifically limit the fine-grained specific attribute annotation data for each common clothing attribute.
[0067] In one embodiment of this application, when training the image-text dual encoder, this application can achieve cross-modal alignment. For example, when the data type of the query data is image-text, this application can extract semantic attribute features from the image-text data through the aforementioned preset image encoder and preset general text encoder, and align them to a specific attribute subspace of shared clothing attributes according to the common clothing attributes to which the semantic attribute features belong, and then encode the global feature vector. For example, "red" is aligned to the color attribute subspace, and "pure cotton" is aligned to the material attribute subspace.
[0068] When training the image encoder, single-modal feature enhancement can be performed. This application allows for self-supervised contrastive learning (e.g., the MoCo paradigm) of images within each feature extraction module or during the pre-training stage. Two views of the same image obtained through different data enhancements are considered positive sample pairs. By modifying the vector element values of the feature vectors corresponding to the two views, the similarity of the encoded vectors of the two views is maximized, where the similarity is less than a first preset threshold. Views of different images are considered negative sample pairs, where the similarity of the views of different images is minimized, where the similarity is greater than a second preset threshold. The first and second preset thresholds are set by the user according to the scenario and are not specifically limited here. This helps the model learn more fundamental attribute features that are insensitive to disturbances (such as lighting and angle changes).
[0069] In another embodiment of this application, a user may upload multiple images at once as query data. In this case, to better extract attribute feature vectors for attribute fusion, the method further includes:
[0070] When the query data includes multiple reference images, one or more specified reference attributes are obtained from the user terminal. These specified reference attributes have a mapping relationship with the reference images and are used to characterize the attribute feature dimension of the attribute feature vector extracted from the corresponding reference image. Based on the specified reference attributes and the set of attribute modules, the corresponding attribute feature vectors in each reference image are determined and input into the downstream attribute fusion processing task.
[0071] In other words, when the server receives multiple reference images uploaded by a user at once, the server can request the user to retrieve a specified reference attribute for each image in the batch, or a specified reference attribute for the entire batch. For example, the user might want to "use..." Figure 1 "style" and "use" Figure 2 "Fabric", at this time the server will use the attribute module set and specified reference attribute to extract only the attribute feature vector corresponding to the specified reference attribute from the corresponding image, such as from... Figure 1 Extract the style vector V_style_A from Figure 2 Extract the fabric vector V_fabric_B). This enables support for multi-image joint feature search, improving the user experience.
[0072] In this embodiment, the server determines the corresponding attribute fusion feature information based on the query semantic information, attribute weight vector, and a pre-built set of attribute modules, specifically including:
[0073] Based on the common clothing attributes corresponding to the query semantic information, the retrieval participation feature extraction modules in the attribute module set are determined. The query data is input into the retrieval participation feature extraction modules to determine one or more corresponding attribute feature vectors based on the feature extraction results. Based on the attribute weight vector and the weighted calculation results of each attribute feature vector, the attribute fusion feature vector is determined, and the weighted attribute feature vector and attribute fusion feature vector corresponding to each attribute feature vector are used as attribute fusion feature information.
[0074] The attribute fusion feature vector is determined based on the weighted calculation results of the attribute weight vector and each attribute feature vector, using one of the following methods:
[0075] The attribute feature vectors are linearly combined based on the attribute weight vectors to calculate the attribute fusion feature vector V_fused = w1·V1 + w2·V2 + ... + w n ·V n , where w n V represents the attribute weight corresponding to the nth common clothing attribute. n The attribute feature vector corresponding to the nth common clothing attribute;
[0076] The attribute weight vector and the feature vector of each attribute are input into the pre-trained adaptive feature selection model to calculate the attribute fusion feature vector V_fused; the adaptive feature selection model includes at least one of the following: attention mechanism and gating network.
[0077] In other words, this application, by parsing the query semantic information, can obtain the common clothing attributes corresponding to the query semantic information, such as the two dimensions of color and material. The server then uses the feature extraction model corresponding to these two dimensions as the feature extraction module for retrieval and processes the query data to obtain the extracted attribute feature vectors. Subsequently, the server uses the obtained attribute weight vector and attribute feature vector to perform weighted calculations according to the common clothing attributes, thereby fusing the attribute feature vectors of the color and material dimensions to obtain the attribute fusion feature vector. Simultaneously, the server will also extract the weighted attribute feature vectors of the color and material dimensions separately, and combine them with the attribute fusion feature vector to form the attribute fusion feature information.
[0078] There are two methods for obtaining the aforementioned attribute fusion feature vector: one is to use linear weighted summation, and the other is to train an adaptive feature selection model. Users can choose according to their actual usage scenario, and this application does not impose specific limitations on this method. The aforementioned common clothing attributes include two dimensions: color and material. This is merely an example; in actual use, different dimensions of common clothing attributes may exist based on the user's input query data, and no specific limitations are imposed here.
[0079] Furthermore, when this application uses only some attributes for fusion (e.g., query A uses color + style, query B uses color + material), even if the basic visual content is similar, the resulting attribute fusion feature vectors V_fused_A and V_fused_B may become too close in the vector space due to the averaging effect or dimensional collapse of the fusion process, making it impossible to accurately distinguish which attribute combination drives the similarity. Therefore, this application also provides the following embodiments:
[0080] The server will obtain the attributes of interest from the common clothing attributes specified in the user's query data according to the above steps, and use the attribute fusion feature vector of the attributes of interest as the basic fusion vector. Then, it will concatenate a preset additional dimension to display the encoded attribute information. The basic fusion vector has a dimension of D (e.g., 512 dimensions), and the number of additional dimensions is N (N is the total number of attributes). The additional dimensions are used to store binary attribute activation masks or attribute weight values, where: the binary mask uses "1" to represent the corresponding attribute activation and "0" to represent inactivation (e.g., [1, 1, 0, ...]); the attribute weight values are normalized values (e.g., [0.6, 0.4, 0, ...]). For example, concatenating an 8-dimensional attribute mask makes the final vector dimension D+8 (e.g., 520 dimensions). By calculating the similarity of the complete vector (basic fusion vector + additional dimensions), the joint distinction between fused content and attribute combination information is achieved. During the model training phase, an attribute difference loss function is introduced to penalize the phenomenon of fusion vectors of different attribute combinations being too close, and then the re-ranking retrieval step S103 is executed.
[0081] Furthermore, this application can avoid using the general independent weighted fusion for semantically strongly related attribute pairs (such as "sleeve type" and "cuff"). A joint feature extraction module can be designed to directly output the combined vector V_sleeve_cuff, or a nonlinear fusion function specifically designed to learn the interaction between the two can be called during fusion, with the result then fused with other attributes. This generates more expressive, specific representations for particular and important attribute combinations, reducing semantic confusion caused by inappropriate linear combinations.
[0082] S103, the server matches several candidate clothing images in the clothing image database based on attribute fusion feature information, retrieval strategies from user terminals and preset ANN algorithms, and generates an image sorting sequence through a preset re-sorting model.
[0083] The retrieval strategies include at least overall fusion feature retrieval, single-dimensional feature retrieval, and multi-path single-dimensional feature retrieval. The default ANN algorithm is the Approximate Nearest Neighbor (ANN) algorithm.
[0084] In this embodiment of the application, the above-mentioned matching of several candidate clothing images in the clothing image database based on attribute fusion feature information, retrieval strategy from the user terminal, and preset ANN algorithm specifically includes:
[0085] The system receives a retrieval strategy from the user terminal and determines the relevant feature vectors of the corresponding interested attributes in the attribute fusion feature information. According to the retrieval strategy, one or more of the interested feature vectors are recombined, and the recombined feature vector group is matched with the feature vectors of clothing images in the clothing image database for similarity. Based on the similarity matching results, candidate clothing image feature vectors are determined, resulting in several candidate clothing images.
[0086] In other words, users can pre-specify search strategies through their user terminals, which can be devices such as mobile phones or computers; this application does not impose specific limitations on this. Search strategies can include overall fusion feature retrieval, single-dimensional feature retrieval, and multi-path single-dimensional feature retrieval. Overall fusion feature retrieval can be understood as performing a search based on attribute fusion feature vectors generated from all common clothing attributes; single-dimensional feature retrieval can be understood as performing a search based on a weighted attribute feature vector of a single common clothing attribute; multi-path single-dimensional feature retrieval can be understood as simultaneously performing searches on the weighted attribute feature vectors of two or more common clothing attributes separately, and then fusing the results of the separate searches.
[0087] According to the above retrieval strategy, the server reorganizes the feature vectors corresponding to the attributes of interest contained in the semantic information of the user query to obtain a set of feature vectors of interest that satisfies the retrieval strategy. For example, in the overall fusion feature retrieval, all feature vectors of interest corresponding to common clothing attributes are added to the set of feature vectors of interest, and feature vectors of common clothing attributes that are not present in the attribute of interest are padded with 1s or all 0s; in the single-dimensional feature retrieval, the feature vectors of interest corresponding to a single common clothing attribute are added to separate sets of feature vectors of interest. Subsequently, the cosine similarity or the inverse of the Euclidean distance is calculated for each feature vector of interest and each clothing image feature vector in the clothing image database to obtain the similarity matching result. The feature vectors of clothing images with similarity greater than a preset threshold are used as candidate clothing image feature vectors, and candidate clothing images are obtained through the correspondence between vectors and images. The preset threshold can be set by the user according to the actual use scenario, and this application does not impose specific limitations on it.
[0088] In this embodiment of the application, the above-mentioned generation of image sorting sequence through a preset reordering model specifically includes:
[0089] The feature vectors of each candidate clothing image and their corresponding attention feature vectors are masked. The candidate clothing image feature vectors, attention feature vectors, masked candidate clothing image feature vectors, and masked attention feature vectors are then concatenated to obtain an initial feature concatenation vector. There is a one-to-one correspondence between the initial feature concatenation vector and the candidate clothing images. This initial feature concatenation vector is input into a preset re-ranking model to score the candidate clothing image feature vectors and determine the recommended score for each candidate clothing image. The labels of each candidate clothing image are then added to the image ranking sequence in descending order of recommended scores.
[0090] In other words, after obtaining candidate clothing images, this application can further mask the feature vectors of the candidate clothing images and their corresponding attention feature vectors using a pre-defined re-ranking model (such as the Rerank model). For example, attention feature vectors (Q_vec_color, Q_vec_style, ...), candidate clothing image feature vectors (C_vec_color, C_vec_style, ...), and masking vectors (Q_mask_color, C_mask_color, ...). Based on the masked vectors, the attention feature vectors, and the candidate clothing image feature vectors, an initial feature concatenation vector is constructed: [Q_vec_color, C_vec_color, Q_mask_color, C_mask_color, Q_vec_style, C_vec_style, Q_mask_style, C_mask_style, ...]. An initial feature concatenation vector corresponds to a candidate clothing image. The aforementioned pre-defined re-ranking model processes this initial feature concatenation vector, combining the focus feature vector and masking results to score the feature vectors of the candidate clothing images, obtaining a recommendation score. The labels of each candidate clothing image are then sorted from highest to lowest recommendation score. The pre-defined re-ranking model can employ a deep network model such as the Transformer.
[0091] S104, the server sends the corresponding recommended clothing images to the user terminal according to the image sorting sequence.
[0092] The user terminal can be understood as a user's mobile phone, computer, or other devices; this application does not specifically limit this. The recommended clothing images are sorted sequentially according to the image sorting sequence and displayed on the user terminal's interface.
[0093] This application can be used for clothing similarity retrieval on e-commerce platforms: users upload reference clothing images and input "find similar styles, but in blue". The NLU module parses the instruction, the system performs color attribute replacement, and returns the results. If a user searches for "long-sleeved dress", the system can use (optional) classification results to first filter out "dress", and then match the "long-sleeved" style vector. It can also be used for designers' inspiration search: designers upload multiple reference images and input "use..." Figure 1 The silhouette and Figure 2 The NLU module parses the instructions, and the system executes a multi-image joint search function to obtain comprehensively matched clothing images as inspiration references. For personalized recommendations and fashion matching: clothing images viewed or liked by the user are used as query data, and attribute vectors such as style and color are extracted separately. Dynamic fusion is then used to form user preferences; then, a similarity search is performed with inventory clothing images to provide users with highly customized recommendation results.
[0094] Furthermore, this application employs a microservice architecture to deploy various feature extraction models, making the impact of adding, deleting, and modifying modules on the overall system controllable. The core retrieval process does not require rewriting, significantly improving the system's flexibility and maintainability. To ensure online stability, a canary release strategy is preferred for deploying new or updated modules. For example, a small amount of online query traffic is first used to verify the effectiveness of the new service. Once the metrics (such as retrieval accuracy and system latency) meet expectations, the traffic is gradually increased until a full replacement or deployment is achieved.
[0095] This application, through the aforementioned scheme, utilizes a pre-built image encoder and an NLU module to simultaneously process user-input image and text information, accurately identifying query intent and attribute parameters. It employs multiple independent feature extraction models to capture different dimensions of clothing features, dynamically meeting diverse retrieval needs, reducing irrelevant feature interference, and improving retrieval efficiency and accuracy. Furthermore, this application supports diverse retrieval strategies, covering various scenario retrieval needs, and flexibly and systematically outputting clothing recommendation results, thus enhancing user experience. It solves the problem of insufficient dynamic adaptability of feature representation and retrieval strategies in current personalized clothing recommendations, significantly improving the accuracy of personalized recommendations and user experience, and meeting the urgent needs of e-commerce and fashion industries for high-quality clothing retrieval.
[0096] Figure 2 A schematic diagram of the structure of an artificial intelligence-based personalized clothing recommendation system provided in this application embodiment is shown below. Figure 2 As shown, the AI-based personalized clothing recommendation system 200 includes:
[0097] The first determining module 201 is used to determine the query semantic information and attribute weight vector corresponding to the query data input by the user, based on a preset query data parsing module. The preset query data parsing module includes at least a preset image encoder and a preset NLU module. The query semantic information includes at least the query intent and query attribute parameters. The second determining module 202 is used to determine the corresponding attribute fusion feature information based on the query semantic information, the attribute weight vector, and a pre-constructed attribute module set. The attribute module set contains feature extraction modules corresponding to multiple preset common clothing attributes. The matching generation module 203 is used to match several candidate clothing images in the clothing image database based on the attribute fusion feature information, the retrieval strategy from the user terminal, and a preset ANN algorithm, and generate an image ranking sequence through a preset re-ranking model. The retrieval strategy includes at least overall fusion feature retrieval, single-dimensional feature retrieval, and multi-path single-dimensional feature retrieval. The sending module 204 is used to send the corresponding recommended clothing images to the user terminal according to the image ranking sequence.
[0098] This application embodiment also provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:
[0099] Based on a pre-built query data parsing module, the query semantic information and attribute weight vector corresponding to the user-input query data are determined. This pre-built query data parsing module includes at least a pre-built image encoder and a pre-built NLU module. The query semantic information includes at least the query intent and query attribute parameters. Based on the query semantic information, attribute weight vector, and a pre-constructed set of attribute modules, corresponding attribute fusion feature information is determined. The attribute module set contains feature extraction modules corresponding to multiple pre-built common clothing attributes. Based on the attribute fusion feature information, the retrieval strategy from the user terminal, and a pre-built ANN algorithm, several candidate clothing images in the clothing image database are matched, and an image ranking sequence is generated through a pre-built re-ranking model. The retrieval strategy includes at least overall fusion feature retrieval, single-dimensional feature retrieval, and multi-path single-dimensional feature retrieval. Based on the image ranking sequence, the corresponding recommended clothing images are sent to the user terminal.
[0100] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0101] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.
[0102] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0103] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A personalized clothing recommendation method based on artificial intelligence, characterized in that, The method includes: Based on a pre-set query data parsing module, the query semantic information and attribute weight vector corresponding to the user-input query data are determined. The pre-set query data parsing module includes at least a pre-set image encoder and a pre-set NLU module. The pre-set NLU module includes a pre-set natural language understanding model in artificial intelligence, capable of parsing the user's natural language to determine the user's focus attributes and other necessary clothing attributes, and adding them to the global feature vector. The query semantic information includes at least the query intent and query attribute parameters. Specifically, after receiving the user-input query data, the data type of the query data is determined. The data type includes at least one of the following: text, image, and image-text. Based on the data type, the pre-set image encoder and / or the pre-set NLU module in the pre-set query data parsing module are invoked to determine the global feature vector corresponding to the query data. The global feature vector is matched with a pre-set historical semantic data list to determine the corresponding query semantic information based on the matching result. The global feature vector is input into a pre-trained weight prediction network model to determine the attribute weight vector based on the model output. The output of the weight prediction network model is a vector w with a fixed dimension, whose dimension is equal to the predefined total number of modules with common clothing attributes. Each element in vector w... This represents the importance weight of the i-th attribute module; Based on the query semantic information, the attribute weight vector, and the pre-constructed attribute module set, the corresponding attribute fusion feature information is determined. The attribute module set contains multiple preset feature extraction modules corresponding to common clothing attributes. Each feature extraction module in the pre-constructed attribute module set corresponds to an independent common clothing attribute, which includes at least one of the following: color, material, style, and type. When only some attributes are used for fusion, the focus attributes in the common clothing attributes specified by the user query data are obtained according to the above steps. The attribute fusion feature vector of the focus attributes is used as the basic fusion vector, and then a preset additional dimension is concatenated to display the encoded attribute information. The basic fusion vector dimension is D, and the number of additional dimensions is N. The additional dimensions are used to store binary attribute activation masks or attribute weight values so as to achieve joint differentiation of fused content and attribute combination information by calculating the similarity of the complete vector. Based on the attribute fusion feature information, the retrieval strategy from the user terminal, and the preset ANN algorithm, several candidate clothing images in the clothing image database are matched, and an image sorting sequence is generated through a preset re-sorting model; wherein, the retrieval strategy includes at least overall fusion feature retrieval, single-dimensional feature retrieval, and multi-path single-dimensional feature retrieval. Based on the image sorting sequence, the corresponding recommended clothing images are sent to the user terminal; Before determining the corresponding attribute fusion feature information based on the query semantic information, the attribute weight vector, and the pre-constructed set of attribute modules, the method further includes: Acquire several feature extraction sample data corresponding to different common clothing attributes; the feature extraction sample data includes at least image-text pair association data, fine-grained specific attribute annotation data, and single-modal feature enhancement data; According to the common clothing attributes, the feature extraction sample data is input into the feature extraction module to be trained, so as to train the image-text dual encoder, attribute text encoder and image encoder in the feature extraction module respectively until the loss function value of the feature extraction module is less than a predetermined value, and the trained feature extraction module is obtained. Then, the trained feature extraction modules corresponding to each common clothing attribute are added to the attribute module set. The feature extraction module was trained using several feature extraction sample data. Specifically, a dual image-text encoder was trained using image-text pair association data, an attribute text encoder was trained using fine-grained attribute-specific labeled data, and an image encoder was trained using single-modal feature enhancement data. The fine-grained attribute-specific labeled data consisted of several fine-grained data points encompassing various dimensions of different common clothing attributes. For example, the fine-grained attribute-specific labeled data for color included at least red, yellow, and pink; and the fine-grained attribute-specific labeled data for material included at least wool and pure cotton. When training the image-text dual encoder, cross-modal alignment is achieved. When the data type of the query data is image-text type, the semantic attribute features in the image-text data are extracted through the above-mentioned preset image encoder and preset general text encoder. Based on the common clothing attribute to which the semantic attribute features belong, they are aligned to the attribute subspace of the special common clothing attribute, and then the global feature vector is encoded. When training the image encoder, for single-modal feature enhancement, self-supervised contrastive learning is performed on the image within each feature extraction module or during the pre-training stage. Two views of the same image obtained through different data enhancements are regarded as positive sample pairs. By modifying the vector element values of the feature vectors corresponding to the two views, the similarity of the encoded vectors of the two views is maximized, and the difference is maximized when the similarity is less than a first preset threshold. Views of different images are regarded as negative sample pairs, and the similarity of the views of different images is minimized, and the difference is minimized when the similarity is greater than a second preset threshold.
2. The personalized clothing recommendation method based on artificial intelligence according to claim 1, characterized in that, The method further includes: When the query data includes multiple reference images, one or more specified reference attributes are obtained from the user terminal; wherein, the specified reference attributes have a mapping relationship with the reference images and are used to characterize the attribute feature dimension of the attribute feature vector extracted from the corresponding reference image; Based on the specified reference attribute and the attribute module set, the corresponding attribute feature vectors in each reference image are determined and input into the downstream attribute fusion processing task.
3. The personalized clothing recommendation method based on artificial intelligence according to claim 1, characterized in that, Based on the query semantic information, the attribute weight vector, and the pre-constructed set of attribute modules, the corresponding attribute fusion feature information is determined, specifically including: Based on the common clothing attributes corresponding to the query semantic information, the retrieval participation feature extraction modules in the attribute module set are determined; The query data is input into the retrieval participation feature extraction module to determine one or more corresponding attribute feature vectors based on the feature extraction results; Based on the weighted calculation results of the attribute weight vector and each attribute feature vector, an attribute fusion feature vector is determined, and the weighted attribute feature vector and the attribute fusion feature vector corresponding to each attribute feature vector are used as the attribute fusion feature information.
4. The personalized clothing recommendation method based on artificial intelligence according to claim 3, characterized in that, Based on the weighted calculation results of the attribute weight vector and each attribute feature vector, the attribute fusion feature vector is determined using one of the following methods: The attribute fusion feature vector is calculated by linearly combining the attribute feature vectors according to the attribute weight vector. ,in For the first Each shared clothing attribute corresponds to a specific attribute weight. No. The attribute feature vectors corresponding to the common clothing attributes; The attribute weight vector and each attribute feature vector are input into a pre-trained adaptive feature selection model to calculate the attribute fusion feature vector. The adaptive feature selection model includes at least one of the following: an attention mechanism and a gating network.
5. The personalized clothing recommendation method based on artificial intelligence according to claim 1, characterized in that, Based on the attribute fusion feature information, the retrieval strategy from the user terminal, and the preset ANN algorithm, several candidate clothing images in the clothing image database are matched, specifically including: Receive the retrieval strategy from the user terminal and determine the respective interest feature vectors of the corresponding interest attributes in the attribute fusion feature information; According to the retrieval strategy, one or more of the aforementioned feature vectors are recombined, and the recombined feature vector group is matched with the feature vectors of clothing images in the clothing image database for similarity. Based on the similarity matching results, candidate clothing image feature vectors are determined, resulting in several candidate clothing images.
6. The personalized clothing recommendation method based on artificial intelligence according to claim 5, characterized in that, The image sorting sequence is generated through a preset re-sorting model, specifically including: The candidate clothing image feature vectors and their corresponding attention feature vectors for each candidate clothing image are masked. The candidate clothing image feature vectors, the attention feature vectors, the masked candidate clothing image feature vectors and the masked attention feature vectors are then concatenated to obtain an initial feature concatenation vector. The initial feature concatenation vector has a one-to-one correspondence with the candidate clothing images. The initial feature concatenation vector is input into the preset reordering model to score the feature vectors of the candidate clothing images and determine the recommended score for each candidate clothing image. According to the recommended scores in descending order, the labels of each candidate clothing image are added to the image sorting sequence in sequence.
7. A personalized clothing recommendation system based on artificial intelligence, characterized in that, The system is capable of executing the personalized clothing recommendation method based on artificial intelligence as described in any one of claims 1-6; the system includes: The first determining module is used to determine the query semantic information and attribute weight vector corresponding to the query data input by the user, based on the preset query data parsing module; wherein, the preset query data parsing module includes at least a preset image encoder and a preset NLU module; the query semantic information includes at least query intent and query attribute parameters; The second determining module is used to determine the corresponding attribute fusion feature information based on the query semantic information, the attribute weight vector, and the pre-constructed attribute module set; the attribute module set includes feature extraction modules corresponding to multiple preset common clothing attributes. The matching generation module is used to match several candidate clothing images in the clothing image database according to the attribute fusion feature information, the retrieval strategy from the user terminal and the preset ANN algorithm, and generate an image ranking sequence through a preset re-ranking model; wherein, the retrieval strategy includes at least overall fusion feature retrieval, single-dimensional feature retrieval and multi-way single-dimensional feature retrieval. The sending module is used to send the corresponding recommended clothing images to the user terminal according to the image sorting sequence.
8. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of executing an artificial intelligence-based personalized clothing recommendation method as described in any one of claims 1-6.