Commodity feature automatic extraction method and system based on large model semantic understanding

Abstract

The invention belongs to the technical field of digital text information processing, and particularly relates to a commodity feature automatic extraction method and system based on large model semantic understanding. Existing commodity information is utilized to construct a training set and a verification set, deep understanding and semantic extraction of the commodity information are realized, during feature proofreading, feature vectors are divided into key vectors and non-key vectors, key parts and non-key parts of commodity features are analyzed emphatically, weights are distributed according to the importance of the key parts and the non-key parts, an extraction result is further optimized, and the accuracy of feature proofreading is improved. The change trend of the feature vectors in the calibration process is dynamically captured, so that an efficient small sample set is constructed, an optimal feature extraction model is screened out, a multi-step extraction mode or a single-step extraction mode can be selected according to the difference of the features during subsequent extraction of new commodity features, the accuracy and efficiency of feature extraction are improved, and the accuracy and efficiency of commodity feature extraction are improved. In practical application, commodity features meeting requirements can be extracted more quickly and more accurately.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital text information processing, and particularly relates to a method and system for automatically extracting commodity features based on large model semantic understanding. Background Art

[0002] With the rapid development of e-commerce platforms, the number of commodity information has increased explosively. How to automatically extract representative feature information from a large amount of commodity descriptions has become one of the key technologies for improving commodity retrieval efficiency, recommendation accuracy, and user experience.

[0003] Traditional commodity feature extraction methods mainly rely on manual rules, keyword matching, or shallow machine learning models. When facing diverse, unstructured, and semantically complex commodity description texts, such methods often have problems such as low recognition accuracy, poor generalization ability, and insufficient semantic understanding of long texts.

[0004] In recent years, with the development of natural language processing technology, pre-trained language models (such as large models like BERT and GPT) have made remarkable breakthroughs in semantic understanding. These large models have advantages such as strong context modeling ability and refined semantic representation, providing a new technical path for semantic feature extraction of commodity texts. However, existing methods based on large models are mostly used for general tasks such as text classification and named entity recognition, and still lack targeted design for feature extraction in the e-commerce commodity scenario. There are still certain challenges especially in dealing with mixed expressions of multiple commodity attributes, colloquial descriptions, and implicit intention parsing.

[0005] In addition, some current semantic extraction schemes have not fully considered the influence of commodity category context on the semantic weight of features, resulting in the possibility that the same keyword may be misjudged as an irrelevant attribute in different categories, thus affecting the overall recognition accuracy. Therefore, there is an urgent need for an automated feature extraction method that can combine the deep semantic understanding ability of large models and at the same time adapt to the commodity category context to achieve higher-quality and more robust commodity structured processing. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for automatically extracting commodity features based on large model semantic understanding, which can improve the accuracy and efficiency of commodity feature extraction.

[0007] The technical solution adopted by the present invention is specifically as follows: A method for automatically extracting commodity features based on large model semantic understanding, including: S1. Commodity information collection: Obtain the commodity information of existing commodities; wherein, the commodity information includes picture information and text book information; S2. Image Semantic Encoding Clustering: Input the image information into a pre-trained large model to obtain the deep features of the image information, and perform clustering processing on the deep features to obtain high-dimensional semantics that match each existing commodity; S3. Model Fine-tuning Proofreading and Optimization: Use the high-dimensional semantics as input to perform multiple rounds of fine-tuning on the large model, and perform a proofreading operation after each round of fine-tuning, so as to obtain a target model for extracting commodity features; S4. Target Commodity Semantic Mapping Feature Matching: Input the image information of the commodity to be extracted into the target model to obtain the high-dimensional semantics of the commodity to be extracted, and then input the high-dimensional semantics into the target model to match the commodity features corresponding to the high-dimensional semantics, and mark them as the reference features of the commodity to be extracted; S5. Feature Confidence Analysis, Extraction and Verification: Aggregate the reference features into a dataset to be verified, and then input the text book information in the dataset to be verified into the target model one by one to obtain the confidence scores of each reference feature, and determine the extraction method of each reference feature according to the confidence scores, and then perform verification extraction to obtain the key features corresponding to each reference feature of the commodity to be extracted.

[0008] In the present invention, preferably, the step of performing clustering processing on the deep features to obtain high-dimensional semantics that match each existing commodity includes: Classify the deep features according to the commodity information, and then perform an offset operation on each deep feature vector in the same category to obtain a number of reference feature vectors; Obtain the reference clustering radius, and based on the reference feature vectors, measure the Euclidean distance between each deep feature vector and it, and mark it as the first deviation; Compare the first deviation with the reference clustering radius. If the first deviation is greater than or equal to the reference clustering radius, classify the corresponding deep feature vector into other categories, and then match its belonging to other categories according to the Euclidean distance from the reference feature vectors in other categories; If the first deviation is less than the reference clustering radius, it indicates that the corresponding deep feature vector and the reference feature vector belong to the same category.

[0009] In the present invention, preferably, the step of performing multiple rounds of fine-tuning on the large model with the high-dimensional semantics as input includes: Construct a training set and a validation set according to each high-dimensional semantics; Obtain the reference proofreading rounds, input the training set within the reference proofreading rounds into the large model, and then input the validation set into the large model to verify the accuracy of the output result of the large model, and mark it as the first proofreading parameter; Compare the first calibration parameter with a preset calibration threshold. If the first calibration parameter is greater than the calibration threshold within the reference calibration rounds, mark the current large model as a model to be calibrated.

[0010] In the present invention, preferably, after the step of marking the current large model as a model to be calibrated when the first calibration parameter is greater than the calibration threshold within the reference calibration rounds, the following steps are further included: Obtain the models to be calibrated for which the first calibration parameter is greater than or equal to the calibration threshold within the reference calibration rounds, and sort the models to be calibrated in descending order according to the magnitude of the first calibration parameter; Mark the model to be calibrated with the highest rank as the model to be output, and then mark the difference between the first calibration parameter lower than the model to be output within the reference calibration rounds and the calibration threshold as the parameter to be evaluated; Obtain the model reference deviation interval, and compare the upper limit value of the model reference deviation interval with the parameter to be evaluated. If the parameter to be evaluated is within the model reference deviation interval, mark the corresponding large model as an outputtable model after the fine-tuning ends; If the parameter to be evaluated is greater than the upper limit value of the model reference deviation interval, continue to perform the calibration operation on the corresponding large model.

[0011] In the present invention, preferably, the step of performing the calibration operation after the calibration ends includes: Obtain the first calibration parameter higher than the parameter corresponding to the model to be output during the fine-tuning process, and mark it as the parameter to be calibrated; Obtain the wear degree of the validation set corresponding to the parameter to be calibrated, mark the difference between the parameter to be calibrated and the parameter corresponding to the model to be output as the amount to be calibrated, and then divide the amount to be calibrated into multiple intervals to be tested according to the wear degree of the validation set; Compare the parameter corresponding to the model to be output with the highest-ranked interval to be tested. If the interval to be tested is lower than the parameter corresponding to the model to be output, compare the parameter to be calibrated with the second-highest-ranked interval to be tested; If the amount to be calibrated is within the interval to be tested corresponding to the parameter of the model to be output, input the corresponding validation set into the large model corresponding to the amount to be calibrated one by one for verification; If the performance parameter of the large model corresponding to the second-highest-ranked interval to be tested is lower than the corresponding parameters of the large models in other intervals to be tested during the verification and calibration process, stop its execution of verification.

[0012] In the present invention, preferably, the step of determining the extraction method of each reference feature according to the confidence score includes: Obtain the confidence parameter values of each reference feature, determine the feature code corresponding to the reference feature according to each confidence parameter, and divide the feature code into a first-order code and a second-order code; Execute the first comparison mode, where the first comparison mode includes: Obtain the first-order codes of all feature codes, and compare the feature codes of the previous to-be-verified extraction image information output result of the to-be-extracted commodity with the feature codes of the current to-be-verified image information output result; If the feature codes before and after extraction are the same, output that the second-order codes are the same, that is, the previous to-be-verified extraction image information output result of the to-be-extracted commodity is the same as the current to-be-verified image information output result; If the feature codes before and after extraction are different, output that the second-order codes are different, that is, the previous to-be-verified extraction image information output result of the to-be-extracted commodity is the same as the current to-be-verified image information output result; According to the output result of the second-order code, execute the corresponding feature extraction method. When the second-order codes are the same, determine whether to verify the output features according to the commodity level of the to-be-extracted commodity. When the second-order codes are different, extract the features with relatively small confidence parameters and output them as non-critical extraction features, and output the other extraction features as critical extraction features.

[0013] In the present invention, preferably, if the second-order codes among multiple features are the same, then sort based on the output results of the confidence parameters of each feature extraction, and mark the feature corresponding to the lowest-ranked confidence parameter as the critical extraction feature. In the case of the same ranking, output the corresponding critical extraction feature according to the commodity level of the to-be-extracted commodity.

[0014] In the present invention, preferably, the target model supports extracting critical features and non-critical features from the to-be-extracted commodity, and the extraction commodity is a commodity for which the target model supported does not support the necessary critical features; The critical extraction features include product name, product model, production manufacturer, etc., and the non-critical extraction features include product packaging, promotional information, etc.

[0015] In the present invention, preferably, the confidence parameters of the extraction method are divided into first-level confidence parameters, second-level confidence parameters, and third-level confidence parameters, and first-level confidence parameters are preferentially supported, and the target model is optimized according to the confidence parameters of the selected features; The optimization does not support the positioning requirements of necessary features. Collect real image data for research, provide weights, and correspondingly generate and output a custom target model.

[0016] The present invention also provides a commodity feature automatic extraction system based on large model semantic understanding for implementing the commodity feature automatic extraction method based on large model semantic understanding, including: A commodity information collection module for obtaining the commodity information of existing commodities, where the commodity information includes picture information and text book information; The picture semantic encoding and clustering module is used to input the picture information into a pre-trained large model, extract corresponding deep features, and perform clustering processing on the deep features to obtain high-dimensional semantics matching each existing commodity; The model fine-tuning and proofreading optimization module is used to take the high-dimensional semantics as input, perform multiple rounds of fine-tuning processing on the large model, and perform proofreading operations after each round of fine-tuning, so as to obtain a target model for extracting commodity features; The commodity semantic mapping and feature matching module is used to input the picture information of the commodity to be extracted into the target model, obtain the corresponding high-dimensional semantics, and match the commodity features corresponding to the high-dimensional semantics through the target model, and mark them as reference features; The feature confidence analysis and verification module is used to summarize the reference features into a dataset to be verified, and input the text book information in the dataset into the target model one by one to calculate the confidence scores of each reference feature; judge the feature extraction method according to the confidence scores, and complete the final verification extraction output of the key features; Among them, the key features at least include product name, product model, and manufacturer, and the non-key features at least include product packaging and promotional information.

[0017] As a preferred technical solution, the picture semantic encoding and clustering module specifically includes: The feature classification unit is used to preliminarily classify the extracted deep features according to commodity information, and perform vector offset processing on each deep feature vector in each category to generate several benchmark feature vectors; The clustering judgment unit is used to obtain a benchmark clustering radius, and take the benchmark feature vector as the center to measure the Euclidean distance between other deep feature vectors and it to obtain a first deviation; The feature classification unit is used to compare the first deviation with the benchmark clustering radius. If the first deviation is greater than or equal to the clustering radius, the corresponding deep feature vector is classified into other categories and the belonging category is matched according to the Euclidean distance; if the first deviation is less than the clustering radius, it is classified into the current category to complete the high-dimensional semantic clustering process.

[0018] The present invention also provides a commodity feature automatic extraction terminal based on large model semantic understanding, including: One or more processors; A storage device on which one or more programs are stored, wherein the one or more programs are configured to be executable by one or more processors, and through the execution of the one or more processors, the commodity feature automatic extraction terminal based on large model semantic understanding executes the commodity feature automatic extraction method based on large model semantic understanding.

[0019] The beneficial effects of the present invention are as follows: By leveraging the features of the large model and using the existing product information to construct a training set and a validation set, the present invention realizes in-depth understanding and semantic extraction of product information. During feature verification, the feature vectors are divided into key vectors and non-key vectors, and the key and non-key parts of the product features are analyzed emphatically, and weights are assigned according to their importance to further optimize the extraction results. By constructing a sliding window, the changing trend of the feature vectors during the verification process is dynamically captured, thereby constructing an efficient small-sample set to help screen out the optimal feature extraction model. Subsequently, when extracting new product features, according to the differences of the features, the multi-step extraction or single-step extraction method can be selected, which improves the accuracy and efficiency of feature extraction. In practical applications, it can extract product features that meet the requirements more quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic flow chart of the method of the present invention; Figure 2 is a schematic diagram of the sub-steps of step S2 of the present invention; Figure 3 is a schematic diagram of the sub-steps of step S3 of the present invention; Figure 4 is a schematic diagram of the sub-steps of step S4 of the present invention; Figure 5 is a schematic diagram of the sub-steps of step S5 of the present invention; Figure 6 is a schematic diagram of the sub-steps of step S6 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention will be further described below in conjunction with specific embodiments. The following embodiments can enable those skilled in the art to understand the present invention more comprehensively, but thus cannot limit the present invention.

[0022] Embodiment 1 This embodiment provides an automatic product feature extraction method based on large model semantic understanding, as Figure 1 shown, including: Step S1: Obtain the product information of existing products, where the product information includes picture information and text book information; In step S1, the collection of product information can be obtained by scraping or collecting from multiple online shopping websites or literature databases. Among them, the picture information can be understood as the appearance of the product, which can be color matching, style, background, etc. The text book information can include the identification text, logo or QR code of the product. The product information covers all the features and necessary features of the product. Among them, the conversion of the image information into the necessary information of the product is consistent with the upper limit of the image information of the product. The corresponding text book information of the product can be converted into multiple corresponding necessary features of the product. To improve the execution efficiency, all the text book information is preferentially parsed through the image information, and then arranged by evaluating the important features of the product from the user's perspective. Among them, the secondary confidence parameter corresponds to the need for secondary judgment. For example, some key features of the product are likely to be different, and the most common ones are determined as size, material, image ratio parameters, etc., which are selected as the first-level feature parameters.

[0023] Step S2: Input the picture information into the pre-trained large model, obtain the deep features of the picture information, and perform clustering processing on the deep features to obtain the high-dimensional semantics matching each existing product. In step S2, after the picture information is input into the pre-trained large model, the output deep features are used as the understanding of the existing products. Specifically, the degree of its understanding of the products can be equated to the first observation result of a person. Through clustering analysis, the parts of a specific picture that may be consistent with other products during the production process are extracted, such as small parts, etc. These small parts can be sold separately by themselves and can also be used as components of large parts. After adding the text book information later, they can be distinguished. The small parts are just an example. Correspondingly, the clustering here can be fine-tuned according to the selection of a small sample set to output an imitation large model. Step S3: Use the high-dimensional semantics as the input, perform multiple rounds of fine-tuning processing on the large model, and perform a proofreading operation after each round of fine-tuning to obtain the target model for extracting product features. In step S3, fine-tuning is a processing method for adjusting the large model. Its purpose is to simulate the deep extraction of picture information and then append it to the large model to shorten the extraction speed of the high-frequency features of these daily products by the large model and improve the user experience. Specifically, a training set and a validation set will be constructed during the fine-tuning process, and both can be obtained by scraping or collecting in the first step. During the proofreading process, it reflects the understanding accuracy of the large model, that is, the output can be compared with the text book information. Step S4: Input the picture information of the product to be extracted into the target model to obtain the high-dimensional semantics of the product to be extracted, and then input the high-dimensional semantics into the target model to match the product features corresponding to the high-dimensional semantics and mark them as the reference features of the product to be extracted. In step S4, during the proofreading process, the initial recognition is to match the product features corresponding to the input picture, and the construction of high-dimensional semantics is also achieved through a similar accumulation method, that is, by fine-tuning the model to train an unexpandable model (custom target model) covering all information, decomposing it into small samples to output these necessary features, and then converting them into important parameters, such as: for subsequent entry features, image segmentation, and quality assessment output. Obviously, the subsequent fine-tuning model optimizes according to the input features to improve the accuracy rate, and will surely output a more efficient target custom model; Step S5: Aggregate the referenceable features into a dataset to be verified, and then input the text book information in the dataset to be verified into the target model one by one to obtain the confidence scores of each referenceable feature, determine the extraction method of each referenceable feature according to the confidence scores, and then perform verification extraction to obtain the key features corresponding to each referenceable feature of the product to be extracted.

[0024] In step S5, input the text book information in the dataset to be verified into the large model one by one, truly simulate the user's perspective, export the corresponding confidence parameters, and clarify the importance and accuracy of the product features according to the confidence scores. The higher the confidence score, the more accurate the corresponding referenceable feature of the product, that is, the more key features, and vice versa are non-key features. The difference in the extraction method lies in the feature parameters, and here the ones with higher confidence scores will be retained as the output results. Also, pay attention to fine-tuning each side of the product as an independent target, and then adjust the combination according to the corresponding product situation. For example: give priority to identifying the main identifiers such as the corresponding trademark, output the comparison parameters, which is combined with the target custom model, and can specify each product as the most matching generated result.

[0025] Embodiment 2 As Figure 2 shown, in this embodiment, the steps of clustering the depth features to obtain high-dimensional semantics matching each existing product include: Step S201: Classify the depth features according to the product information, and then perform an offset operation on each depth feature vector in the same category to obtain several benchmark feature vectors; In step S201, the depth features are classified according to their corresponding product information. The depth features in the same category (such as image information and expression information may match the information that has been collected). Here, using the correlation between the depth features, perform an offset operation to level each feature. It can be understood that the extraction essentially is just a way to present the image information at the front end. Combine some images that are different but have a high similarity together. Here, the operation will be reduced to a set, so that it will not cause the load of the large model and will not lead to frequent adjustment of the large model parameters to generate more high-frequency and low-precision extraction schemes.

[0026] Step S202: obtaining a reference clustering radius, and taking the reference feature vector as a reference, calculating the Euclidean distance between each depth feature vector and the reference feature vector, and marking it as a first deviation; In step S202, the benchmark clustering radius is a range of values ​​calculated by the TD&R model. This is also to reserve an adjustment margin for the benchmark feature vector. It refers to the maximum difference between different vectors when performing deep feature clustering. Here, it is ensured that its value range is a defined interval or a specific value, and the radius size is affected by the selected parameters. If the benchmark feature vector is not unique, multiple benchmark feature vectors are required. Specifically, it is suitable for all situations where the coordination between images and information is relatively high, that is, the image cannot be quickly identified to its precise category and needs to be used in combination with other conditions.

[0027] Step S203: Compare the first deviation with the reference clustering radius. If the first deviation is greater than or equal to the reference clustering radius, classify the corresponding deep feature vector into other categories, and then match it to other categories according to the Euclidean distance between the first deviation and the reference feature vector in other categories. If the first deviation is smaller than the reference clustering radius, it indicates that the corresponding depth feature vector and the reference feature vector belong to the same category; In step S203, the Euclidean distance is used to calculate the offset between the deep feature vector and the baseline feature vector to extract whether there is a difference. Correspondingly, the large model tends to be combined with the deep feature vector and then adjusted. Here, the deviation is the key parameter to measure the classification accuracy of the feature vector. The classification effect of the baseline feature vector can be adjusted by evaluating the size of the deviation.

[0028] Example 3 like Figure 3 As shown, in this embodiment, the steps of performing multiple rounds of fine-tuning on the large model with high-dimensional semantics as input include: Step S301: constructing a training set and a validation set according to each high-dimensional semantics; In step S301, a training set is constructed that includes a portion of corresponding high-dimensional semantics. This is the same as the execution method of the traditional training set. Cross-validation is performed, specifically including: deep feature classification, baseline feature offset and classification judgment and other operations. The verification set includes input image information, which can be one or more images. It reflects the understanding of the image information by the large model during the fine-tuning process. This execution method is commonly used in the image modeling process, that is, when the sample size is insufficient, the current output content will be used as subsequent verification data, so that the execution feedback for better multi-experience can be improved.

[0029] Step S302: Obtain the reference calibration rounds, input the training set within the reference calibration rounds into the large model, then input the validation set into the large model, verify the accuracy of the output results of the large model, and mark it as the first calibration parameter; In step S302, the training set and the validation set within the reference calibration rounds are successively input into the large model (multiple times) to improve the large model's ability to understand image information and correspondingly reduce the probability of errors occurring during its application. The marked calibration parameters will be used to participate in corresponding multiple loops to reduce the error results in the validation set, thereby assisting in further adjusting the large model, such as the number of loops and using the incorrect data as the training set again.

[0030] Step S303: Compare the first calibration parameter with a preset calibration threshold. Within the reference calibration rounds, when the first calibration parameter is greater than the calibration threshold, mark the current large model as a model to be calibrated; In step S303, the threshold is determined according to the measurement ability of the large model. Here, it can be understood as the maximum loss that the large model can accept in a short period of time. After exceeding this value, it is necessary to perform the next round of large-loop parameter adjustment and the coordination operation between feature vectors. Specifically, it can be shown according to the comprehensive execution situation, such as user feedback, product failure rate, etc. Here, the parameters will be selected according to the situation of the large model within its selectable range. At the same time, for the large model that has reached the critical value and has been collected and fine-tuned accordingly, record its corresponding end time. For small samples that have reached the critical value, directly generate a high-frequency target model.

[0031] Embodiment 4 As Figure 4 shown, in this embodiment, after the step of marking the current large model as a model to be calibrated when the first calibration parameter is greater than the calibration threshold within the reference calibration rounds, it further includes: Step S401: Obtain the models to be calibrated whose first calibration parameter is greater than or equal to the calibration threshold within the reference calibration rounds, and sort each model to be calibrated from large to small according to the magnitude of the first calibration parameter; Step S402: Mark the model to be calibrated with the highest rank as the model to be output, and then mark the difference between the first calibration parameter lower than the model to be output and the calibration threshold within the reference calibration rounds as the parameter to be evaluated; Step S403: Obtain the model reference deviation interval, compare the upper limit value of the model reference deviation interval with the parameter to be evaluated. If the parameter to be evaluated is within the model reference deviation interval, mark the corresponding large model as an outputable model after fine-tuning; If the parameter to be evaluated is greater than the upper limit value of the model reference deviation interval, continue to perform the calibration operation on the corresponding large model; In step S403, fine-tuning and proofreading operations are respectively performed to further determine the ranking of the model to be proofread. First, a reference deviation range is determined, which includes a set of values related to the parameter to be evaluated. By determining the position of the parameter to be evaluated and calculating its deviation from the upper limit value of the model reference deviation range.

[0032] In steps S401 to S403, the first proofreading parameter is compared with a preset proofreading threshold through the model to be proofread with the highest rank and is ranked among the top in the model to be proofread. We determine the output of the proofreading model by marking the model that is higher than or equal to the preset proofreading threshold as the model to be output.

[0033] Embodiment 5 As Figure 5 shown, in this embodiment, the steps of performing the proofreading operation after the proofreading is completed include: Step S501: Obtain the first proofreading parameter that is higher than the corresponding parameter of the model to be output during the fine-tuning process and mark it as the parameter to be proofread; Step S502: Obtain the wear degree of the validation set corresponding to the parameter to be proofread, mark the difference between the parameter to be proofread and the corresponding parameter of the model to be output as the quantity to be proofread, and then divide the quantity to be proofread into multiple intervals to be tested according to the wear degree of the validation set; In steps S501 to S502, the wear degree is determined. According to the difference between the parameter to be proofread and the corresponding parameter of the model to be output, combined with the determination of the wear degree of the validation set associated with the parameter to be proofread, the position of the quantity to be proofread in the interval to be tested of the wear degree of the validation set is judged.

[0034] Step S503: Compare the corresponding parameter of the model to be output with the interval to be tested with the highest rank. And in the case where the corresponding parameter of the model to be output is lower than the interval to be tested, compare the parameter to be proofread with the interval to be tested with the second highest rank; If the quantity to be proofread is within the interval to be tested of the corresponding parameter of the model to be output, input the corresponding validation set one by one into the large model corresponding to the quantity to be proofread for verification; If the performance parameter of the large model corresponding to the interval to be tested with the second highest rank is lower than the corresponding parameters of the large models of other intervals to be tested during the verification and proofreading process, stop its execution of verification; In step S503, through the proofreading operation, the parameter to be proofread is compared with the intervals to be tested with the highest and second highest ranks, and then the parameter to be proofread is determined. The main basis is to evaluate whether the quantity to be proofread is ranked second in the validation set, and we stop its verification comparison with the high-pillow proofreading parameter in order to save unnecessary resources.

[0035] As Figure 6As shown in the figure, the specific steps are as follows. The steps of determining the extraction method of each reference feature according to the confidence score include: Step S601: Obtain the confidence parameter values of each reference feature, determine the feature code corresponding to the reference feature according to each confidence parameter, and divide the feature code into a first-order code and a second-order code; In step S601, after initially determining the confidence score and feature parameters, when extracting features, there may be commodities with the same style but different colors. Here, we can assign a vector to represent the corresponding commodity. For example, during the production process, a corresponding feature code will be assigned to this situation. This code will include the confidence parameter and the extraction parameter, which is convenient for subsequent retrieval and use. The marking of the vector is not unique, which is the basis for subsequent comparison and will not be restricted by the type of commodity. We can just adjust the corresponding vector during subsequent updates; Step S602: Execute the first comparison mode, where the first comparison mode includes: Obtain the first-order codes of all feature codes, and compare the feature code of the output result of the previous to-be-verified extraction image information of the commodity to be extracted with the feature code of the current to-be-verified image information output result; If the feature codes before and after extraction are the same, output that the second-order codes are the same, that is, the output result of the previous to-be-verified extraction image information of the commodity to be extracted is the same as the output result of the current to-be-verified image information; If the feature codes before and after extraction are different, output that the second-order codes are different, that is, the output result of the previous to-be-verified extraction image information of the commodity to be extracted is the same as the output result of the current to-be-verified image information; Step S603: According to the output result of the second-order code, execute the corresponding feature extraction method. When the second-order codes are the same, determine whether to verify the output features according to the commodity level of the commodity to be extracted. When the second-order codes are different, extract the features with relatively small confidence parameters and output them as non-critical extraction features, and output the other extraction feature as a critical extraction feature.

[0036] In steps S602 to S603, when determining the comparison of feature parameters, this part of the comparison result is determined as the second-order code, and the commodity features that are relatively urgent for specific commodities are retained for output. The rest can be output as optional display content, mainly based on some large commodities (commodities restricted by the volume of containers) to define their key parameters.

[0037] By using the standard data of the critical extraction features, and then comparing the extracted commodity features with the critical features more precisely, various parameters can be determined to evaluate the tuning direction of the target large model.

[0038] The target model supports extracting key features and non-key features from the products to be extracted. The products to be extracted are those for which the target model does not support the necessary key features. The key extraction features include product name, product model, manufacturer, etc., and the non-key extraction features include product packaging, promotional information, etc.

[0039] In this embodiment, based on the features of the large model, the existing product information is used to construct a training set and a validation set to achieve in-depth understanding and semantic extraction of product information. During feature verification, the feature vectors are divided into key vectors and non-key vectors, and the key and non-key parts of the product features are analyzed emphatically, and weights are assigned according to their importance to further optimize the extraction results. By constructing a sliding window, the changing trend of the feature vectors during the verification process is dynamically captured, so as to construct an efficient small sample set to help screen out the optimal feature extraction model. Subsequently, when extracting new product features, according to the differences of the features, the multi-step extraction or single-step extraction method can be selected to improve the accuracy and efficiency of feature extraction. In practical applications, the product features that meet the requirements can be extracted more quickly and accurately.

[0040] Example 6 This embodiment provides a system for automatically extracting product features based on large model semantic understanding. This system is built on a Python programming framework and a deep learning platform (such as TensorFlow or PyTorch), and combines a multi-modal semantic modeling and feature confidence analysis mechanism to automatically extract key features from product text and image information. Specifically, it includes the following modules: Product information collection module: This module batch grabs the image information of products and the corresponding product text descriptions (including product name, brand description, packaging description, etc.) by accessing the product management database API. The system supports image formats such as JPEG and PNG, and standardizes the text information and encodes it into a UTF-8 format text stream.

[0041] Image semantic encoding and clustering module: This module inputs the collected product images into a large model based on the CLIP (Contrastive Language–Image Pre-training) structure. After extracting the visual depth features of each image, the system initially groups them according to the product categories, and performs vector offset processing on the depth feature vectors within each group, and calculates the Euclidean distance between each image vector. A clustering radius is constructed based on the category center vector. If the deviation distance of an image vector is less than the clustering radius, it is included in the current clustering; otherwise, it is reclassified by comparing other clustering centers. Finally, the high-dimensional semantic labels of each product image are output.

[0042] Model Fine-tuning and Proofreading Optimization Module: This module receives the high-dimensional semantics of the clustered images, combines the labeled features of each category of products, constructs a training set and a validation set, and is used to fine-tune the parameters of the initial large model. After each round of fine-tuning, the results of the validation set are compared with the true labels, the accuracy is calculated, the first proofreading parameter is generated, and compared with the preset proofreading threshold. After multiple rounds of proofreading, when the output accuracy of the model meets the proofreading conditions, it is marked as the target product feature extraction model.

[0043] Commodity Semantic Mapping and Feature Matching Module: This module inputs the target product image to be extracted into the calibrated target model and outputs its corresponding high-dimensional semantic label. The system then compares this label with the existing semantic clustering information, matches the historical product semantic set most relevant to the target product, and extracts the labeled graphic and text features therein as the reference features of this product.

[0044] Feature Confidence Analysis and Verification Module: This module evaluates all reference features item by item to obtain their confidence parameter values. The system maps the confidence values into an encoded form, divided into first-order encoding and second-order encoding, and performs encoding consistency judgment between the front and back images. If the second-order encodings of the corresponding features of two images are the same, they are marked as stable features, and it is determined whether to output them as key features according to the product level; if they are different, the confidence parameter values are compared, and the feature with a higher confidence value is extracted as the key feature, and the other is marked as a non-key feature.

[0045] Output Result Module: The system outputs the finally extracted key features and non-key features to the product management system in JSON format. The key feature fields at least include product name, product model, and manufacturer. The non-key features include packaging styles, promotional statements, etc.

[0046] The above embodiments are only the preferred embodiments of the present invention, and do not limit the technical solutions of the present invention. Any equivalent transformations and improvements made based on the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A method for automatically extracting commodity features based on large model semantic understanding, characterized in that Including: S1. Collection of product information: Obtain the product information of existing products; among them, the product information includes picture information and text book information; S2. Picture semantic encoding clustering: Input the picture information into a pre-trained large model, obtain the deep features of the picture information, and perform clustering processing on the deep features to obtain high-dimensional semantics matching each existing product; S3. Model fine-tuning, proofreading and optimization: Use the high-dimensional semantics as the input, perform multiple rounds of fine-tuning processing on the large model, and perform a proofreading operation after each round of fine-tuning, so as to obtain a target model for extracting product features; S4. Target product semantic mapping feature matching: Input the picture information of the product to be extracted into the target model to obtain the high-dimensional semantics of the product to be extracted, and then input the high-dimensional semantics into the target model to match the product features corresponding to the high-dimensional semantics, and mark them as the reference features of the product to be extracted; S5. Feature confidence analysis, extraction and verification: Summarize the reference features into a data set to be verified, and then input the text book information in the data set to be verified into the target model one by one to obtain the confidence scores of each reference feature, and determine the extraction method of each reference feature according to the confidence scores, and then perform verification extraction to obtain the key features corresponding to each reference feature of the product to be extracted.

2. The automatic extraction method of product features based on large model semantic understanding according to claim 1, wherein In the step S2, the step of performing clustering processing on the deep features to obtain high-dimensional semantics matching each existing product includes: Classify the deep features according to the product information, and then perform an offset operation on each deep feature vector in the same category to obtain a number of reference feature vectors; Obtain the reference clustering radius, and measure the Euclidean distance between each deep feature vector and it based on the reference feature vector, and mark it as the first deviation amount; Compare the first deviation amount with the reference clustering radius. If the first deviation amount is greater than or equal to the reference clustering radius, classify the corresponding deep feature vector into other categories, and then match its belonging to other categories according to the Euclidean distance from the reference feature vectors in other categories; If the first deviation amount is less than the reference clustering radius, it indicates that the corresponding deep feature vector and the reference feature vector belong to the same category.

3. The method for automatically extracting product features based on large model semantic understanding according to claim 2, wherein In the step S3, the step of performing multiple rounds of fine-tuning processing on the large model with the high-dimensional semantics as the input includes: Construct a training set and a validation set according to each high-dimensional semantics; Obtain the reference proofreading round number, input the training set within the reference proofreading round number into the large model, and then input the validation set into the large model to verify the accuracy of the output result of the large model, and mark it as the first proofreading parameter; Compare the first proofreading parameter with a preset proofreading threshold, and within the reference proofreading round number, if the first proofreading parameter is greater than the proofreading threshold, mark the current large model as a model to be proofread.

4. The method for automatically extracting commodity features based on large model semantic understanding according to claim 3, wherein After the step of marking the current large model as a model to be proofread when the first proofreading parameter is greater than the proofreading threshold within the reference proofreading round number, it further includes: Obtain the models to be proofread whose first proofreading parameter is greater than or equal to the proofreading threshold within the reference proofreading round number, and sort each model to be proofread from large to small according to the size of the first proofreading parameter; Mark the model to be proofread with the highest rank as the model to be output, and then mark the difference between the first proofreading parameter lower than the model to be output and the proofreading threshold within the benchmark proofreading rounds as the parameter to be evaluated; Obtain the model reference deviation interval, and compare the upper limit value of the model reference deviation interval with the parameter to be evaluated. If the parameter to be evaluated is within the model reference deviation interval, mark the corresponding large model as an outputtable model after the fine-tuning is completed; If the parameter to be evaluated is greater than the upper limit value of the model reference deviation interval, continue to perform the proofreading operation on the corresponding large model.

5. The method for automatically extracting product features based on large model semantic understanding according to claim 1, wherein In step S5, the steps of determining the extraction method of each reference feature according to the confidence score include: Obtain the confidence parameter values of each reference feature, determine the feature encoding corresponding to the reference feature according to each confidence parameter, and divide the feature encoding into first-order encoding and second-order encoding; Execute the first comparison mode, where the first comparison mode includes: Obtain the first-order encoding of all feature encodings, and compare the feature encoding of the output result of the previous image information to be verified for the product to be extracted with the feature encoding of the current image information to be verified; If the feature encodings before and after extraction are the same, output that the second-order encodings are the same, that is, the output result of the previous image information to be verified for the product to be extracted is the same as the output result of the current image information to be verified; If the feature encodings before and after extraction are different, output that the second-order encodings are different, that is, the output result of the previous image information to be verified for the product to be extracted is the same as the output result of the current image information to be verified; Execute the corresponding feature extraction method according to the output result of the second-order encoding. When the second-order encodings are the same, determine whether to verify the output feature according to the product level of the product to be extracted. When the second-order encodings are different, extract the feature with a relatively small confidence parameter and output it as a non-critical extraction feature, and output the other extraction feature as a critical extraction feature.

6. The automatic extraction method of product features based on large model semantic understanding according to claim 5, characterized in that, If the second-order encodings of multiple features are the same, sort based on the output results of the confidence parameters of each feature extraction, mark the feature corresponding to the confidence parameter with the lowest rank as the critical extraction feature, and output the corresponding critical extraction feature according to the product level of the product to be extracted in the case of the same rank.

7. The method for automatically extracting product features based on large model semantic understanding according to claim 1, wherein, The critical extraction features at least include the product name, product model, and production manufacturer, and the non-critical extraction features at least include the product packaging and promotional information.

8. A commodity feature automatic extraction system based on large model semantic understanding, which is used to implement the commodity feature automatic extraction method based on large model semantic understanding according to any one of claims 1-7, and is characterized in that, Include: A commodity information collection module for obtaining the commodity information of existing commodities, where the commodity information includes picture information and text book information; A picture semantic encoding and clustering module for inputting the picture information into a pre-trained large model, extracting the corresponding deep features, and performing clustering processing on the deep features to obtain the high-dimensional semantics matching each existing commodity; A model fine-tuning and proofreading optimization module for using the high-dimensional semantics as input, performing multiple rounds of fine-tuning processing on the large model, and performing proofreading operations after each round of fine-tuning, so as to obtain a target model for extracting commodity features; A commodity semantic mapping and feature matching module, which is used to input the picture information of the commodity to be extracted into the target model to obtain the corresponding high-dimensional semantics, and match the commodity features corresponding to the high-dimensional semantics through the target model, and mark them as reference features; A feature confidence analysis and verification module, which is used to summarize the reference features into a data set to be verified, and input the text book information in the data set into the target model one by one to calculate the confidence scores of each reference feature; judge the feature extraction method according to the confidence scores, and complete the final verification extraction output of the key features; Among them, the key features at least include product name, product model, and production manufacturer, and the non-key features at least include product packaging and promotional information.

9. The automatic product feature extraction system based on large model semantic understanding according to claim 8, wherein, The picture semantic encoding and clustering module specifically includes: A feature classification unit, which is used to preliminarily classify the extracted deep features according to commodity information, and perform vector offset processing on each deep feature vector in each category to generate a number of benchmark feature vectors; A clustering judgment unit, which is used to obtain the benchmark clustering radius, and calculate the Euclidean distance between other deep feature vectors and it with the benchmark feature vector as the center to obtain the first deviation; A feature classification unit, which is used to compare the first deviation with the benchmark clustering radius. If the first deviation is greater than or equal to the clustering radius, the corresponding deep feature vector is classified into other categories and the belonging category is matched according to the Euclidean distance; if the first deviation is less than the clustering radius, it is classified into the current category to complete the high-dimensional semantic clustering process.

10. An automatic commodity feature extraction terminal based on large model semantic understanding, characterized in that, Including: At least one processor; And a memory communicatively connected to the processor; Among them, executable instructions are stored in the memory, and when the executable instructions are executed by the processor, the processor can execute the commodity feature automatic extraction method based on large model semantic understanding according to any one of claims 1 to 7.

Images