Adaptive jewelry image data processing method and apparatus

Abstract

The embodiment of the application provides a kind of adaptive jewelry image data processing method and device, method includes: through target detection and cutting pre-processing to the jewelry picture uploaded by user, and through digital asset management system, similar graph search and high-definition image replacement are carried out, input quality is improved, then, whether the search range can be limited by judging whether the similar search result of original drawing can utilize commodity model number;If not, the jewelry category is intelligently identified by multi-modal large language model, the multi-modal vector model after the jewelry field sample fine-tuning is used to feature coding to cutting graph, generate field adaptive image vector, and accordingly similar recall is carried out in vector database, jewelry category filtering, model number deduplication and multi-dimensional reordering of fusion visual similarity, image quality and commodity attribute are sequentially executed to the recall result, output final search result, the efficiency and precision of jewelry picture search can be improved by the application.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to an adaptive jewelry image data processing method and apparatus. Background Technology

[0002] With the rapid development of jewelry retail, e-commerce, and digital operations, users are increasingly seeking to search for the same or similar jewelry items by taking or uploading photos. However, user-uploaded images often come from diverse sources, including real-life photos, social media screenshots, and images of models wearing the jewelry. This results in significant differences in image quality, background complexity, shooting angle, and lighting conditions, posing a considerable challenge to accurate retrieval.

[0003] Existing image-based product search technologies have several shortcomings when applied to the jewelry industry: First, inconsistent image quality, with issues like low resolution, background interference, or incomplete subject matter directly impacting feature extraction and matching accuracy; second, most solutions employ a single image similarity retrieval path, making it difficult to balance precise matching of identical items with recommendations of similar items; third, general pre-trained image vector models are mostly built based on natural scenes, limiting their ability to characterize the unique visual features of jewelry (such as metallic luster, gemstone inlay, and fine craftsmanship), resulting in insufficient recall or a high false positive rate; fourth, jewelry category recognition largely relies on manual rules or pre-labeling, resulting in low automation and difficulty adapting to large-scale retrieval needs; finally, the same product often corresponds to multiple images in the database, leading to duplicate display of search results and negatively impacting user experience.

[0004] Therefore, there is an urgent need for an adaptive jewelry image data processing method that can improve the efficiency and accuracy of jewelry image retrieval. Summary of the Invention

[0005] To address the problems in the existing technology, this application provides an adaptive jewelry image data processing method and apparatus, which can improve the efficiency and accuracy of jewelry image retrieval.

[0006] To solve at least one of the above problems, this application provides the following technical solution: In a first aspect, this application provides an adaptive jewelry image data processing method, including: The system acquires jewelry images uploaded by users, performs target detection and cropping on the main jewelry area of ​​the images, determines the cropped image containing only the target jewelry, inputs the jewelry images into a preset product image library for similarity retrieval, if there are product images with similarity higher than a preset threshold and the product images are associated with a product model number, then the product model number of the product image is recorded; if no product model number is detected, then the system performs category recognition on the jewelry images based on a preset multimodal large language model to determine the corresponding jewelry category information. The cropped image is encoded according to a defined domain image vector embedding model to determine the corresponding image vector. The image vector is then input into a vector database for similarity retrieval to determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on a pre-trained multimodal image vector basic model based on a defined jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. Based on the jewelry category information, the candidate product set is filtered by category, and the filtered results are deduplicated based on the product module number. The deduplicated candidate product set is then reordered according to image vector similarity, image quality index, and product attribute matching degree. A preset number of jewelry products are then output to determine the corresponding jewelry search results.

[0007] Furthermore, before inputting the jewelry image into a preset product image library for similarity retrieval, the following steps are included: The jewelry image is input into a preset high-definition asset database for similarity retrieval. If an asset image that meets the criteria is found, the jewelry image is replaced with the asset image, and a subsequent similarity retrieval is performed based on the replaced jewelry image. The asset image that meets the criteria includes an asset image whose similarity to the original jewelry image is higher than a preset threshold and whose quality meets the preset high-definition criteria. The replaced jewelry image is subjected to target detection and target cropping to determine the corresponding new cropped image containing the target jewelry. The original cropped image is replaced with the new cropped image, and subsequent image vector encoding is performed based on the replaced cropped image.

[0008] Furthermore, the step of performing category recognition on the jewelry image based on a preset multimodal large language model to determine the corresponding jewelry category information includes: The jewelry image is obtained as input, and cross-modal feature extraction is performed on the input image according to a preset multimodal large language model to determine the structured semantic representation corresponding to the image. The multimodal large language model is obtained by pre-training on a natural image-text pairing dataset. The classification module performs category discrimination on the structured semantic representation according to the set classification module to determine the corresponding jewelry category information. The classification module is obtained after adaptive fine-tuning training for the jewelry field. The fine-tuning training is based on the jewelry dataset, which includes at least one of wearing scene images, tiled images, and local detail images, and the annotation information covers at least one of bracelets, necklaces, pendants, rings, and earrings.

[0009] Furthermore, the step of performing domain-adaptive fine-tuning training on the pre-trained multimodal image vector base model based on a set jewelry domain sample dataset includes: Construct a sample dataset for the jewelry domain, including: Collect jewelry images containing product templates as raw data, and automatically construct positive sample pairs based on the product templates. The positive sample pairs are multiple images of the same template from different angles or in different scenes. Negative sample pairs are constructed based on different product templates, wherein the negative sample pairs are combinations of images with different randomly selected templates; Sample pairs with similar appearances but different product models are introduced as interference samples to enhance the model's ability to identify fine-grained differences; The jewelry domain sample dataset is input into a pre-trained multimodal image vector base model. A loss function is constructed based on the sample pair label information. The loss function includes a positive sample loss term for constraining the vector distance between positive sample pairs and a negative sample loss term for constraining the vector distance between negative sample pairs. The parameters of the pre-trained multimodal image vector base model are iteratively optimized using the backpropagation algorithm until the loss function converges, thereby determining the corresponding domain image vector embedding model.

[0010] Further, the step of inputting the image vector into a vector database for similarity retrieval to determine the corresponding candidate product set includes: Detect whether the image vector is associated with a recorded product template; If a recorded product model number exists, the vector database is pre-indexed based on the product model number to determine the corresponding pre-screened subset after model number filtering, and the image vector is searched for similarity based on the pre-screened subset to determine the corresponding candidate product set. If no recorded product template number exists, the image vector is input into the vector database, and a large-scale data retrieval is performed within the vector database to conduct a similarity search on the image vector and determine the corresponding candidate product set.

[0011] Furthermore, the step of filtering the candidate product set based on the jewelry category information and deduplicating the filtered results based on the product module number includes: Obtain the product attribute information of each candidate product in the candidate product set, match the product attribute information with the jewelry category information, retain the candidate products that match successfully, and remove the candidate products that do not match successfully. For the candidate products retained after matching, their corresponding product template numbers are extracted. For multiple candidate products with the same product template number, one candidate product is selected as the representative result of the product template number according to the preset deduplication rules.

[0012] Furthermore, the process of re-sorting the deduplicated candidate product set based on image vector similarity, image quality indicators, and product attribute matching degree, and outputting a preset number of sorted jewelry products to determine the corresponding jewelry search results includes: For the deduplicated candidate product set, image quality is evaluated using a preset image quality evaluation model to determine the corresponding product image quality index. Based on the product image quality index, combined with image vector similarity and product attribute matching degree, the product images in the candidate product set are re-ranked, and the product images with the highest ranking are output according to a preset number to determine the corresponding jewelry search results.

[0013] Secondly, this application provides an adaptive jewelry image data processing apparatus, comprising: The jewelry image processing module is used to acquire jewelry images uploaded by users, perform target detection and target cropping on the main jewelry area of ​​the jewelry images, determine the corresponding cropped image containing only the target jewelry, input the jewelry images into a preset product image library for similarity retrieval, if there are product images with similarity higher than a preset threshold and the product images are associated with product model numbers, then the product model number of the product images is recorded; if no product model number is detected, then the jewelry images are classified based on a preset multimodal large language model to determine the corresponding jewelry category information. The jewelry product set determination module is used to encode the cropped image according to a set domain image vector embedding model, determine the corresponding image vector, input the image vector into a vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on a pre-trained multimodal image vector basic model based on a set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. The jewelry search result determination module is used to filter the candidate product set based on the jewelry category information, remove duplicates from the filtered results based on the product module number, re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, output a preset number of sorted jewelry products, and determine the corresponding jewelry search results.

[0014] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the adaptive jewelry image data processing method.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the adaptive jewelry image data processing method.

[0016] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the adaptive jewelry image data processing method.

[0017] As can be seen from the above technical solution, this application provides an adaptive jewelry image data processing method and apparatus. It improves input quality by performing target detection and cropping preprocessing on user-uploaded jewelry images, and then performing similar image retrieval and high-definition image replacement through a digital asset management system. Next, based on the similarity retrieval results of the original image, it determines whether the retrieval scope can be limited using product templates. If not, it intelligently identifies jewelry categories using a multimodal large language model, and uses a multimodal vector model fine-tuned with jewelry domain samples to perform feature encoding on the cropped image, generating a domain-adaptive image vector. Based on this, it performs similarity recall in the vector database, sequentially performing jewelry category filtering, template deduplication, and multi-dimensional re-sorting that integrates visual similarity, image quality, and product attributes, outputting the final retrieval result. This improves the efficiency and accuracy of jewelry image retrieval. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the adaptive jewelry image data processing method in this application embodiment; Figure 2 This is a structural diagram of the adaptive jewelry image data processing device in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device in the embodiments of this application.

[0020] Figure label: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0023] Considering the current issues of inconsistent image quality and limited ability of existing image vector extraction models to accurately extract the unique visual features of jewelry during image retrieval, this application provides an adaptive jewelry image data processing method and apparatus. This method improves input quality by performing target detection and cropping preprocessing on user-uploaded jewelry images, followed by similar image retrieval and high-resolution image replacement through a digital asset management system. Then, based on the similarity retrieval results of the original image, it determines whether the search scope can be limited using product templates. If not, a multimodal large language model intelligently identifies the jewelry category. Using a multimodal vector model fine-tuned with jewelry domain samples, feature encoding is performed on the cropped image to generate domain-adaptive image vectors. These vectors are then used for similarity recall in a vector database. The recall results are then subjected to jewelry category filtering, template deduplication, and multi-dimensional re-ranking based on visual similarity, image quality, and product attributes, ultimately outputting the final retrieval result. This improves the efficiency and accuracy of jewelry image retrieval.

[0024] To improve the efficiency and accuracy of jewelry image retrieval, this application provides an embodiment of an adaptive jewelry image data processing method, see [link to embodiment]. Figure 1 The adaptive jewelry image data processing method specifically includes the following: Step S101: Obtain the jewelry image uploaded by the user, perform target detection and target cropping on the main jewelry area of ​​the jewelry image, determine the corresponding cropped image containing only the target jewelry, input the jewelry image into a preset product image library for similarity retrieval, if there is a product image with similarity higher than a preset threshold and the product image is associated with a product model number, then record the product model number of the product image; if no product model number is detected, then perform category recognition on the jewelry image based on a preset multimodal large language model to determine the corresponding jewelry category information; Optionally, in this embodiment, the image acquisition and preprocessing step S1.

[0025] The system acquires the original jewelry images uploaded by the user, performs object detection processing on the original jewelry images, identifies the main body area of ​​the jewelry, and crops the identified main body area to generate a cropped image for subsequent feature extraction and retrieval.

[0026] Specifically, based on pre-trained convolutional neural network models (such as YOLO, Faster R-CNN, etc.), trained on a large amount of jewelry image data, the model can accurately identify the main jewelry area (such as a ring, a necklace, or a pair of earrings) in an image. The model outputs the bounding box position of the jewelry and automatically crops the original image based on the detected bounding box, generating a "cropped image" containing only the main jewelry element. This effectively removes background interference, irrelevant objects, and redundant information, allowing subsequent image feature extraction and similarity comparison to focus more on the structure, material, and design details of the jewelry itself.

[0027] Optionally, in this embodiment, a similar image retrieval and high-definition replacement step S2 is also included for the "jewelry image" and "cropped image" obtained in the previous step.

[0028] Input the original jewelry image into the Digital Asset Management System (DAM) to perform a similar image search: - When an asset image with a similarity to the original jewelry image greater than or equal to a preset threshold is found, and the asset image meets the preset high-definition quality conditions, the original jewelry image is replaced with the asset image. - Re-perform the cropping process based on the replaced high-resolution image, generate a new cropped image, and replace the original cropped image.

[0029] - If no asset image is found, the jewelry search will still use the original jewelry image and the cropped image.

[0030] The aforementioned high-definition replacement mechanism improves the image quality used for subsequent retrieval.

[0031] Optionally, in this embodiment, the original image similarity retrieval and model number determination step S3 is included.

[0032] Perform a similar image search based on the original jewelry image (or the replaced high-resolution jewelry image): - When a similar image with a similarity greater than or equal to a preset threshold is found, determine whether the similar image is associated with product model information; - If similar images are associated with product model numbers, then subsequent searches will be limited to the products corresponding to those model numbers. - If no similar images that meet the criteria are found, or if the similar images are not associated with the template number information, proceed to the next step.

[0033] Specifically, the similarity search is performed between jewelry images and a preset product image library. The search method can be an image matching algorithm based on traditional features (such as SIFT, ORB) or deep features (such as feature vectors extracted by CNN), which calculates the similarity between the uploaded image and each product main image or detail image in the image library.

[0034] Based on a preset similarity threshold (e.g., 90% or higher), the system determines whether highly similar existing product images have been found. If product images with a similarity higher than this threshold exist in the search results, it indicates that the jewelry uploaded by the user is very likely to highly match or be the same as a known product in the company's product database.

[0035] Further examination is needed to determine if the highly similar product image is associated with a product template number. A product template number is an internal code used by a company to uniquely identify a jewelry product, typically linked to the product's SKU and style number. If a template number is associated, the system will record it. Identifying candidate template numbers at the initial stage significantly narrows the subsequent search scope to the corresponding product and its variations (such as different colors or materials), greatly improving the efficiency and accuracy of exact match searches and avoiding unnecessary generalized searches across the entire database.

[0036] Optionally, in this embodiment, the intelligent identification step S4 is for jewelry categories.

[0037] When the search scope is not limited by the model number, the original jewelry image is analyzed based on a multimodal large language model to automatically identify the category information of the jewelry image; Jewelry categories include, but are not limited to: bracelets, necklaces, pendants, rings, and earrings.

[0038] Specifically, if no image with a similarity higher than the threshold is found in the similarity search of the product image library, or if the found similar images are not associated with a valid product template (for example, some images in the image library may lack template labels or are not product images), the system will perform image category recognition based on a multimodal large language model (MLLM).

[0039] The multimodal large language model possesses visual understanding and semantic association capabilities. The system inputs the original uploaded image into the model, which analyzes the image content and, combined with its internal knowledge of jewelry categories (such as "bracelet," "necklace," "pendant," "ring," and "earrings"), predicts and outputs the most likely jewelry category information. This is useful when a specific product cannot be directly located through image similarity, at least determining the general category of the jewelry. The obtained category information provides key constraints for subsequent search filtering. For example, if identified as "ring," subsequent vector retrieval and result filtering can exclude other categories such as necklaces and earrings, significantly improving the relevance of search results and reducing computational overhead.

[0040] Step S102: Encode the cropped image according to the set domain image vector embedding model to determine the corresponding image vector, input the image vector into the vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on the pre-trained multimodal image vector basic model based on the set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. Optionally, in this embodiment, the domain-adaptive vector embedding generation step S5 is included.

[0041] The cropped image is subjected to vector feature encoding, specifically including: 1. Select a pre-trained multimodal image vector model as the base model; 2. Fine-tune the basic model based on a sample dataset from the jewelry domain to generate a domain-adaptive vector embedding model; 3. Encode the cropped image using a domain-adaptive vector embedding model to generate the corresponding image vector.

[0042] The jewelry-related sample dataset includes at least the following: - Image samples of the same jewelry item taken from different shooting angles and in different wearing scenarios. - Negative sample pairs between different jewelry items; - A pair of jewelry samples that look similar but have different product models.

[0043] Through the above fine-tuning training, the generated image vectors are made to better match the visual feature distribution in the jewelry field, thereby improving the recall capability of similar products.

[0044] Specifically, the accuracy and discriminative power of image vector encoding directly determine the quality of subsequent retrieval, and its implementation mainly relies on a deep model specifically optimized for the jewelry domain, the "Domain Image Vector Embedding Model".

[0045] Based on the image vector base model pre-trained on a public multimodal dataset, the base model is subjected to domain-adaptive fine-tuning to adapt to the unique visual patterns of jewelry, such as metallic luster, gemstone setting structure, texture details and shape contours.

[0046] The fine-tuning training process uses a jewelry domain sample dataset and designs three key sample pairs to improve the model's discrimination and generalization capabilities: Firstly, images of the same jewelry item under different shooting angles, lighting conditions, and wearing scenarios constitute positive sample pairs. This forces the model to learn to extract stable core features related to the product's identity from the ever-changing appearance, thereby enhancing the model's robustness to changes in viewing angle and background interference. Secondly, images of different jewelry items form negative sample pairs, which are used to increase the distance between different items in the vector space, enhance the model's discrimination ability, and avoid feature confusion between different items; Third, images that are visually highly similar but correspond to different product models constitute interference sample pairs. These samples are particularly important in the jewelry field because many products may be similar in design elements (such as similar pendant shapes) but are actually different styles. By enabling the model to distinguish these "confusing items," the model's sensitivity to subtle differences (such as size and craftsmanship details) can be effectively improved, thereby more accurately characterizing similar but not identical products in vector space and reducing false recalls.

[0047] After fine-tuning on this dataset, the image vectors output by the model more deeply encode the specialized visual features of the jewelry field.

[0048] Optionally, in this embodiment, the large-scale retrieval step (S6) of the vector database is included. The image vector is input into the vector database to perform a similarity search and obtain a preset number of candidate products. Similarity retrieval is achieved based on cosine similarity or equivalent vector distance metrics to enable high recall retrieval of a large number of candidate products.

[0049] Specifically, the image vectors generated by the aforementioned model are used for efficient retrieval. These vectors are then input into a vector database (such as one built using engines like FAISS or Milvus). The database pre-stores vector indices obtained by encoding standard images (typically high-resolution asset images) of each item in the entire product library using the same model.

[0050] During retrieval, the system calculates the similarity between the input vector and all vectors in the database (usually using metrics such as cosine similarity or Euclidean distance). The vector database, through an efficient approximate nearest neighbor search algorithm, can recall a certain number of items most similar to the input vector from hundreds of thousands or even millions of vectors in a very short time, forming an initial candidate item set.

[0051] The process achieved the goal of "large-scale recall". Its advantage lies in its extremely fast speed and ability to perform global comparison based on deep visual features. It effectively covers potential products that are the same or highly similar to the standard image but differ greatly from the standard image due to differences in shooting angle, lighting, background, etc., thus significantly improving the recall rate.

[0052] Meanwhile, before this step of retrieval, if the image vector is associated with the product model number obtained in step S3 above, then before large-scale data retrieval from the vector database, a pre-retrieval of the vector database is performed based on the product model number to obtain a vector subset that is only for the range of that product model number. During subsequent retrieval, the input image vector is only matched with this pre-retrieved subset, which greatly improves the efficiency and accuracy of retrieval.

[0053] Step S103: Based on the jewelry category information, filter the candidate product set by category, and deduplicate the filtered results based on the product module number. Re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, and output the preset number of jewelry products after sorting to determine the corresponding jewelry search results.

[0054] Optionally, in this embodiment, the multi-level filtering and module deduplication step S7 is included.

[0055] The following processes are performed sequentially on the candidate product set: 1. Based on the identified jewelry category information, filter the candidate product set by category; 2. Based on the product module number information, the filtered candidate product set is deduplicated so that only one representative result is retained for the same module number product.

[0056] Specifically, based on the jewelry category information (such as bracelets, necklaces, pendants, rings, earrings, etc.) identified in the previous steps, the candidate product set obtained from the large-scale recall of the vector database is filtered.

[0057] Iterate through each product item in the candidate set and match its category attributes (marked in the backend) with the target category identified by the query image. Only products with completely identical categories or that conform to a preset mapping relationship (for example, when identified as "pendant," products under the "necklace" category that contain pendant components may also be retained, depending on business rules) are retained, while the rest are filtered out.

[0058] After completing the category filtering, we further addressed the issue of duplicate product displays.

[0059] In actual product databases, the same piece of jewelry (corresponding to a unique product model number or SKU) often has multiple images, such as a main image, detailed images, images showing how it looks when worn, and images from different angles. During the vector retrieval stage, these different images may be retrieved simultaneously due to similar visual features, resulting in multiple entries pointing to the same product in the final results, affecting result diversity and browsing efficiency.

[0060] The system groups the filtered product set according to its unique "product module". Within each product group corresponding to a module, it typically retains the best or most representative product based on a predefined strategy (e.g., selecting the product item corresponding to the image with the highest similarity to the query image vector, or selecting the product item with the highest image quality score), while the remaining products are removed.

[0061] Optionally, in this embodiment, the reordering and result output step S8 is included.

[0062] The deduplicated candidate product set is reordered based on a preset sorting rule; The sorting rules must include at least one or more of the following factors: - Image vector similarity; - Image quality metrics; - Product attribute matching degree.

[0063] Output the preset number of sorted jewelry items as the final search results.

[0064] Specifically, after filtering and deduplication, the resulting product set already possesses good relevance and uniqueness. However, to further optimize the ranking quality, a multi-dimensional re-ranking is implemented. A comprehensive scoring model is constructed, taking into account the following core factors: Visual similarity: This is the cosine similarity score between the query image vector and the product image vector, which is the basis for measuring appearance similarity.

[0065] Image quality metrics: These assess the sharpness, composition, brightness, and contrast of product images. High-quality product images clearly showcase details, enhancing user trust and purchase intention.

[0066] Product attribute matching degree: Where possible, combine the attribute preferences implicit in the user query or identified through other means (such as material preference for "gold", style preference for "simple", inlay for "diamonds", etc.) and calculate the matching degree with the product attribute tags.

[0067] By weighted fusion of scores from the above multiple dimensions (the weights can be adjusted according to business objectives; for example, visual similarity weight can be increased when emphasizing similarity searches, and image quality weight can be increased when emphasizing display effects), a final comprehensive score is calculated for each candidate product, and the products are sorted in descending order based on this score.

[0068] This example demonstrates how this embodiment optimizes input quality through image preprocessing, high-definition replacement, and model number determination. It utilizes a multimodal model finely tuned for the jewelry field to generate image vectors for vector retrieval, and combines category filtering, model number deduplication, and multi-dimensional reordering to output the final result. This improves the accuracy and recall rate of image-based jewelry search, and is applicable to various scenarios in digital asset management.

[0069] As described above, the adaptive jewelry image data processing method provided in this application can improve input quality by performing target detection and cropping preprocessing on jewelry images uploaded by users, and by performing similar image retrieval and high-definition image replacement through a digital asset management system. Then, based on the similarity retrieval results of the original image, it is determined whether the retrieval scope can be limited by the product model number. If not, the jewelry category is intelligently identified through a multimodal large language model. The cropped image is feature-encoded using a multimodal vector model fine-tuned by samples from the jewelry domain to generate a domain-adaptive image vector. Based on this, similarity recall is performed in the vector database. The recall results are then sequentially processed by jewelry category filtering, model number deduplication, and multi-dimensional re-sorting that integrates visual similarity, image quality, and product attributes to output the final retrieval results. This improves the efficiency and accuracy of jewelry image retrieval.

[0070] In one embodiment of the adaptive jewelry image data processing method of this application, it may further include the following: Step S201: Input the jewelry image into a preset high-definition asset database for similarity retrieval. If an asset image that meets the conditions is found, replace the jewelry image with the asset image and perform subsequent similarity retrieval based on the replaced jewelry image. The asset image that meets the conditions includes an asset image whose similarity to the original jewelry image is higher than a preset threshold and whose quality meets the preset high-definition conditions. Step S202: Perform target detection and target cropping on the replaced jewelry image to determine the corresponding new cropped image containing the target jewelry. Replace the original cropped image with the new cropped image and perform subsequent image vector encoding based on the replaced cropped image.

[0071] Optionally, in this embodiment, after image cropping and before formal retrieval, a high-definition image replacement step is performed to improve image input quality and enhance the reliability of subsequent data.

[0072] Specifically, the core is "replacing inferior with superior", which involves inputting the original jewelry images uploaded by users into a preset high-definition asset database (DAM) for similarity retrieval. This database stores a large number of high-resolution, clean-background, subject-focused, and color-accurate standard or promotional images of jewelry that have been professionally photographed and post-processed.

[0073] The system calculates the visual similarity between the uploaded image and each high-resolution asset image in the database, and sets a preset similarity threshold (usually high, such as above 85%) as the "matching" threshold. Simultaneously, it evaluates whether candidate asset images meet preset high-resolution conditions, which may include: resolution higher than a specific pixel value (e.g., 1024x768), image clarity indicators (e.g., no blur), background purity, color saturation, and whether it is an officially certified main image, etc.

[0074] When one or more asset images are retrieved that simultaneously meet the criteria of "similarity above the threshold" and "quality meeting the high-definition requirements", the system will select the one with the best quality (or the one with the highest overall similarity and quality score) and use it to replace the user's original uploaded jewelry image.

[0075] After obtaining the high-resolution image of the replaced jewelry: First, replace the original jewelry images with high-resolution jewelry images.

[0076] Second, after performing target detection and cropping on the high-definition jewelry image, the original cropped image is replaced with the high-definition cropped image.

[0077] From this point on, all subsequent operations that rely on cropped images, especially the crucial step of image vector encoding (i.e., converting an image into a feature vector using a domain-adaptive model), will be based on this higher-quality image input.

[0078] Through step S202, this embodiment replaces user images of complex origins and varying quality with high-quality standard asset images as much as possible, laying a solid foundation for subsequent high-quality jewelry image retrieval.

[0079] In one embodiment of the adaptive jewelry image data processing method of this application, it may further include the following: Step S301: Obtain the jewelry image as input, extract cross-modal features from the input image according to the preset multimodal big language model, and determine the structured semantic representation corresponding to the image. The multimodal big language model is obtained by pre-training on a natural image-text pairing dataset. Step S302: According to the set classification module, the structured semantic representation is classified to determine the corresponding jewelry category information. The classification module is obtained after adaptive fine-tuning training for the jewelry domain. The fine-tuning training is based on the jewelry dataset. The jewelry dataset includes at least one of wearing scene image, tiled image and local detail image, and the annotation information covers at least one of bracelet, necklace, pendant, ring and earring.

[0080] Optionally, in this embodiment, the jewelry images to be retrieved are precisely classified into specific categories (such as bracelets, necklaces, pendants, etc.) to support subsequent product filtering and retrieval optimization.

[0081] Specifically, the image of the jewelry to be identified is obtained as input. Then, the input image is fed into a pre-defined multimodal large-scale language model. This model has been pre-trained using massive amounts of natural image-text pairing data, possessing powerful cross-modal understanding and representation capabilities, and can extract deep visual semantics from images. Through forward computation of the model, the image is mapped into a structured semantic representation. This representation not only contains the image's features in a general visual space but also forms a semantic embedding corresponding to the text description, providing a rich and structured feature foundation for subsequent category discrimination.

[0082] The extracted structured semantic representations are input into a specially designed classification module. The classification module has been fine-tuned for adaptation to the vertical field of jewelry recognition.

[0083] Training utilizes a jewelry-specific dataset containing multi-class scene images, such as wearing scene images (simulating real-life wearing conditions), tiled images (showing the overall appearance and shape of the item), and detailed images (highlighting fine features such as texture and setting techniques). These samples are all accurately labeled with category tags, covering major jewelry types such as bracelets, necklaces, pendants, rings, and earrings. Through end-to-end or hierarchical fine-tuning on this domain dataset, the classification module can adapt to the unique visual patterns and morphological differences of jewelry, thereby significantly improving accuracy in category identification.

[0084] Through step S302, this embodiment realizes end-to-end automated category identification. The category identification information is used for category-level filtering in the subsequent retrieval process, effectively narrowing the retrieval scope.

[0085] In one embodiment of the adaptive jewelry image data processing method of this application, it may further include the following: Step S401: Construct a sample dataset for the jewelry domain, including: Collect jewelry images containing product templates as raw data, and automatically construct positive sample pairs based on the product templates. The positive sample pairs are multiple images of the same template from different angles or in different scenes. Negative sample pairs are constructed based on different product templates, wherein the negative sample pairs are combinations of images with different randomly selected templates; Sample pairs with similar appearances but different product models are introduced as interference samples to enhance the model's ability to identify fine-grained differences; Step S402: Input the jewelry domain sample dataset into the pre-trained multimodal image vector base model, and construct a loss function based on the sample pair label information. The loss function includes a positive sample loss term for constraining the vector distance between positive sample pairs and a negative sample loss term for constraining the vector distance between negative sample pairs. Step S403: Iteratively optimize the parameters of the pre-trained multimodal image vector base model using the backpropagation algorithm until the loss function converges, and determine the corresponding domain image vector embedding model.

[0086] Optionally, in this embodiment, this step constructs a high-quality jewelry domain sample dataset, and then optimizes the general pre-trained model based on the dataset to finally obtain a domain vector embedding model specifically for jewelry image retrieval.

[0087] Specifically, a dedicated sample dataset for the jewelry category is constructed. The dataset uses images of jewelry with clearly defined product codes as its raw data foundation.

[0088] Positive sample pairs are constructed based on the same product model number, automatically selecting multiple images of the same jewelry piece from different shooting angles, wearing scenarios, or lighting conditions. The aim is to enable the model to learn the invariant features of the same jewelry piece under different visual representations.

[0089] Negative sample pairs are created by randomly selecting images of different product models, forcing the model to distinguish the differences between different styles of jewelry.

[0090] To further enhance the model's ability to identify at a finer level, sample pairs that are visually similar but have different actual product models are introduced as "interference samples." These samples help the model learn to capture more subtle differences in structure, material, or process, thereby reducing mismatches caused by similar appearances in actual searches.

[0091] Secondly, the pre-trained multimodal image vector base model is fine-tuned based on the constructed dataset. The training process is based on the core idea of ​​contrastive learning.

[0092] Specifically, after inputting sample data into the model to obtain initial vector representations, a loss function is constructed based on the label relationships of sample pairs. This loss function contains two key parts: a positive sample loss term, which narrows the distance between vectors corresponding to different images of the same product in the feature space; and a negative sample loss term, which widens the distance between vectors corresponding to different product images. By optimizing this loss function, the model is guided to learn an embedding space that can better represent the unique visual features of the jewelry field (such as metallic luster, gemstone cut, chain link structure, etc.).

[0093] Finally, the model parameters are iteratively optimized using the backpropagation algorithm. In multiple training iterations, the model continuously adjusts its internal parameters to minimize the aforementioned loss function. The training process ends when the loss function stabilizes and converges. The resulting model is a domain-adaptive image vector embedding model, inheriting the powerful feature extraction capabilities of general pre-trained models while deeply integrating prior knowledge from the jewelry domain, enabling it to generate image vectors that more accurately reflect the essential characteristics of jewelry products.

[0094] Through step S403, this embodiment successfully breaks free from the bias of the general model towards natural scenes and instead forms a feature representation space that is sensitive to key attributes such as jewelry material, craftsmanship, and shape, thereby improving the accuracy of subsequent vector retrieval steps.

[0095] In one embodiment of the adaptive jewelry image data processing method of this application, it may further include the following: Step S501: Detect whether the image vector is associated with a recorded product template; Step S502: If there is a recorded product model number, the vector database is pre-indexed based on the product model number to determine the corresponding pre-screened subset after model number filtering, and the image vector is searched for similarity based on the pre-screened subset to determine the corresponding candidate product set. Step S503: If there is no recorded product model number, the image vector is input into the vector database, and a large-scale data retrieval is performed within the vector database to perform similarity retrieval on the image vector and determine the corresponding candidate product set.

[0096] Optionally, in this embodiment, this step is an optional step before performing large-scale data retrieval in the vector library based on image vectors.

[0097] In this step, the system determines whether the currently generated image vector has been associated with a product template number recorded in the database. This association originates from previous similarity image retrieval and template number determination steps. For example, when a user-uploaded original image is identified as highly similar to a known product with a valid template number, its corresponding image vector will carry that template number information. The purpose of this detection is to determine whether there is a clear "exact match" clue in this search, thus influencing the subsequent search strategy.

[0098] If a recorded product template number is detected, the pre-indexed precise retrieval mode based on the template number is activated.

[0099] Using the product template number as a key index, a fast pre-query is performed on the vector database to filter out all product vector entries marked with the same template number, forming a "pre-screening subset." This subset is strictly limited to products associated with the target template number and may include different angle images or versions of the same product. Subsequently, the system calculates the similarity between the current image vector and each vector in this subset only, thereby determining the candidate product set. This significantly reduces the retrieval space, concentrating computational resources on high-probability matching regions. Its core objective is to achieve precise same-product location, significantly improving retrieval efficiency and accuracy, and effectively avoiding large-scale, inefficient similarity calculations on irrelevant products.

[0100] If no valid product model association is detected, the system will switch to the standard vector similarity retrieval mode.

[0101] At this point, the current image vector is directly input into the vector database, and similarity calculation and ranking are performed within the full product vector space of the database to complete large-scale data retrieval. By performing nearest neighbor search in the high-dimensional vector space, a wide range of candidate products similar in appearance, style, or visual features can be recalled, thereby covering users' needs for "similar product recommendations" or "style retrieval" and ensuring the system's recall rate and flexibility.

[0102] Through step S503, this embodiment successfully achieved adaptive traffic distribution of the retrieval strategy, organically integrating the two paradigms of "precise retrieval" and "fuzzy retrieval".

[0103] In one embodiment of the adaptive jewelry image data processing method of this application, it may further include the following: Step S601: Obtain the product attribute information of each candidate product in the candidate product set, match the product attribute information with the jewelry category information, retain the candidate products that match successfully, and remove the candidate products that do not match successfully. Step S602: For the candidate products retained after matching, extract their corresponding product template numbers. For multiple candidate products with the same product template number, select one candidate product as the representative result of the product template number according to the preset deduplication rule.

[0104] Optionally, in this embodiment, there is a category filtering step.

[0105] First, the system retrieves the candidate product set obtained in the vector database retrieval step. Each candidate product in this set is pre-associated with structured product attribute information, one of the key attributes being the "product category tag," such as "bracelet," "necklace," and "ring." The system then matches the jewelry category information of the user-uploaded images, previously identified through a multimodal large language model, with the category tag of each candidate product. The matching process employs either exact matching or fuzzy matching based on semantic similarity. For candidate products that match successfully—meaning their category tag is consistent with or highly relevant to the identified user intent category—the system retains them. For products that do not match successfully—meaning their category does not match or their relevance is too low—they are removed from the candidate set.

[0106] This step implements category-level filtering, which can effectively narrow the search scope and exclude a large number of products that are irrelevant to the user's query intent, thereby improving the efficiency of subsequent processing and the relevance of results, and avoiding interference from irrelevant results caused by cross-category recall due to vector similarity retrieval.

[0107] Optionally, in this embodiment, a deduplication step is included.

[0108] The system extracts product templates from the retained candidate products. Each piece of jewelry typically corresponds to a unique product number or template in the database, but multiple records may exist due to different shooting angles, scenes, or image quality. The system identifies and aggregates all candidate products with the same product template, forming several template groups. Then, based on preset deduplication rules, the system selects the most representative candidate product from each template group. Typical deduplication rules include: selecting the product with the highest image quality score in the group (e.g., the best overall evaluation based on image resolution, clarity, and lighting uniformity), or selecting the product with the highest similarity to the user's query image vector, or prioritizing selection based on product status (e.g., whether it's the main image, whether it's listed, or whether it's on promotion).

[0109] Through step S602, this embodiment successfully combines category filtering and deduplication to screen the matched candidate product set, retaining the closest and highest quality products.

[0110] In one embodiment of the adaptive jewelry image data processing method of this application, it may further include the following: Step S701: For the deduplicated candidate product set, perform image quality evaluation using a preset image quality evaluation model to determine the corresponding product image quality index; Step S702: Based on the product image quality index combined with image vector similarity and product attribute matching degree, re-rank each product image in the candidate product set, and output the top-ranked product images according to a preset number to determine the corresponding jewelry retrieval results.

[0111] Optionally, in this embodiment, there is an image quality-aware reordering step.

[0112] The image quality assessment model is invoked to automatically analyze the quality of the main image or display image associated with each candidate product in the dataset. Built on computer vision technology, the model quantifies multiple objective quality dimensions of an image, such as sharpness (whether it is blurry), contrast (whether the light and dark areas are distinct), brightness (whether the exposure is appropriate), and background complexity (whether the background is clean and whether the subject stands out). The model comprehensively calculates these dimensions to generate a unified and comparable image quality score for each product image.

[0113] Next, intelligent sorting based on multi-factor fusion is performed.

[0114] The quality assessment results obtained above are then organically combined with the other two core factors: —Image vector similarity (i.e., the degree of similarity between the user query image and the product image in the deep learning feature space, representing visual similarity); — Matching degree with product attributes (i.e., the consistency between the user's potential intent or the attributes identified by image recognition and the product's labeled attributes, such as material, style, etc.); A weighted summation or other multi-objective decision-making algorithm is used to calculate a final comprehensive ranking score for each candidate product.

[0115] Image vector similarity ensures the visual relevance of search results, product attribute matching enhances semantic and demand-level accuracy, and image quality metrics further optimize the presentation and professionalism of the results list.

[0116] Finally, the system selects a number of products with the highest overall ranking scores based on a preset number (such as Top 10) and outputs the final jewelry search results list to the user.

[0117] Through step S702, this embodiment effectively avoids the problem of high-quality products being buried due to inconsistent image quality in the product database, or low-quality images affecting user judgment.

[0118] To improve the efficiency and accuracy of jewelry image retrieval, this application provides an embodiment of an adaptive jewelry image data processing apparatus for implementing all or part of the aforementioned adaptive jewelry image data processing method. See [link to embodiment]. Figure 2 The adaptive jewelry image data processing device specifically includes the following components: The jewelry image processing module 10 is used to acquire jewelry images uploaded by users, perform target detection and target cropping on the main jewelry area of ​​the jewelry image, determine the corresponding cropped image containing only the target jewelry, input the jewelry image into a preset product image library for similarity retrieval, if there is a product image with similarity higher than a preset threshold and the product image is associated with a product model number, then the product model number of the product image is recorded; if no product model number is detected, then the jewelry image is classified based on a preset multimodal large language model to determine the corresponding jewelry category information. The jewelry product set determination module 20 is used to encode the cropped image according to a set domain image vector embedding model, determine the corresponding image vector, input the image vector into a vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on a pre-trained multimodal image vector basic model based on a set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. The jewelry retrieval result determination module 30 is used to filter the candidate product set based on the jewelry category information, remove duplicates from the filtered results based on the product module number, re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, output the sorted preset number of jewelry products, and determine the corresponding jewelry retrieval results.

[0119] As described above, the adaptive jewelry image data processing device provided in this application embodiment can improve input quality by performing target detection and cropping preprocessing on jewelry images uploaded by users, and by performing similar image retrieval and high-definition image replacement through a digital asset management system. Then, based on the similarity retrieval results of the original image, it is determined whether the retrieval scope can be limited by the product model number. If not, the jewelry category is intelligently identified through a multimodal large language model. The cropped image is feature-encoded using a multimodal vector model fine-tuned by samples from the jewelry domain to generate a domain-adaptive image vector. Based on this, similarity recall is performed in the vector database. The recall results are then sequentially processed by jewelry category filtering, model number deduplication, and multi-dimensional re-sorting that integrates visual similarity, image quality, and product attributes to output the final retrieval result. This improves the efficiency and accuracy of jewelry image retrieval.

[0120] To further illustrate this solution, this application also provides a specific application example of using the aforementioned adaptive jewelry image data processing device to implement the adaptive jewelry image data processing method, which specifically includes the following: The complete flowchart of this method is as follows: flowchart TD A [Start] --> B [User-uploaded original image of jewelry] B -->C [Object Detection and Automatic Cropping] Generate cropped image] C --> D [Perform similar graph retrieval in the DAM system] D -->|Similarity ≥ Preset Threshold| E{Is it a high-resolution asset image} D -->|Similarity <Preset Threshold| H E -->|Yes| F[Replace the original image with a high-resolution asset image] And recut E -->|No| H F --> G [Get the high-resolution original image + cropped image] G --> H [Perform similar image search on the original image] H -->I{Does it exist?} Images with similarity ≥ a preset threshold I -->|Yes| J{Are similar images? Related product templates} I -->|No| M J -->|Yes| K[Determine the module range] [Limited search product set] J -->|No| M M --> N [Based on a multimodal large language model] Identify jewelry categories K -->O [Perform a cutout on the pattern] Vector feature encoding N -->O O -->P [Based on Domain Adaptive Vector Model] [Generate image vector] P --> Q [Vector database similarity retrieval] Large-scale recall of candidate products] Q --> R [Filter based on jewelry category] R --> S [Deduplication based on product template] S --> T [Multi-dimensional reordering] Visual similarity / Image quality / Product attributes T --> U [Output the Top N Jewelry Items] U -->V [End] From a hardware perspective, in order to improve the efficiency and accuracy of jewelry image retrieval, this application provides an embodiment of an electronic device for implementing all or part of the adaptive jewelry image data processing method, wherein the electronic device specifically includes the following: The system comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between the adaptive jewelry image data processing method and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the adaptive jewelry image data processing method in the present embodiment, and the contents of the embodiments of the adaptive jewelry image data processing method are incorporated herein, and repeated details will not be described again.

[0121] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0122] In practical applications, parts of the adaptive jewelry image data processing method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0123] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0124] Figure 3This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 3 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 3 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0125] In one embodiment, the adaptive jewelry image data processing method function can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control: Step S101: Obtain the jewelry image uploaded by the user, perform target detection and target cropping on the main jewelry area of ​​the jewelry image, determine the corresponding cropped image containing only the target jewelry, input the jewelry image into a preset product image library for similarity retrieval, if there is a product image with similarity higher than a preset threshold and the product image is associated with a product model number, then record the product model number of the product image; if no product model number is detected, then perform category recognition on the jewelry image based on a preset multimodal large language model to determine the corresponding jewelry category information; Step S102: Encode the cropped image according to the set domain image vector embedding model to determine the corresponding image vector, input the image vector into the vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on the pre-trained multimodal image vector basic model based on the set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. Step S103: Based on the jewelry category information, filter the candidate product set by category, and deduplicate the filtered results based on the product module number. Re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, and output the preset number of jewelry products after sorting to determine the corresponding jewelry search results.

[0126] As described above, the electronic device provided in this application improves input quality by performing target detection and cropping preprocessing on jewelry images uploaded by users, and by performing similar image retrieval and high-definition image replacement through a digital asset management system. Then, based on the similarity retrieval results of the original image, it determines whether the retrieval scope can be limited by the product model number. If not, it intelligently identifies the jewelry category through a multimodal large language model, uses a multimodal vector model fine-tuned by jewelry domain samples to perform feature encoding on the cropped image, generates a domain-adaptive image vector, and performs similarity recall in the vector database accordingly. The recall results are then subjected to jewelry category filtering, model number deduplication, and multi-dimensional re-sorting that integrates visual similarity, image quality, and product attributes to output the final retrieval results. This improves the efficiency and accuracy of jewelry image retrieval.

[0127] In another embodiment, the adaptive jewelry image data processing method can be configured separately from the central processing unit 9100. For example, the adaptive jewelry image data processing method can be configured as a chip connected to the central processing unit 9100, and the function of the adaptive jewelry image data processing method can be realized through the control of the central processing unit.

[0128] like Figure 3 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 3 All components shown; in addition, the electronic device 9600 may also include Figure 3 For components not shown, please refer to existing technologies.

[0129] like Figure 3 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0130] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0131] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0132] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0133] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0134] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which is the same as in a conventional mobile communication terminal.

[0135] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is also coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0136] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the adaptive jewelry image data processing method with a server or client as the execution subject in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the adaptive jewelry image data processing method with a server or client as the execution subject in the above embodiments. For example, when the processor executes the computer program, it implements the following steps: Step S101: Obtain the jewelry image uploaded by the user, perform target detection and target cropping on the main jewelry area of ​​the jewelry image, determine the corresponding cropped image containing only the target jewelry, input the jewelry image into a preset product image library for similarity retrieval, if there is a product image with similarity higher than a preset threshold and the product image is associated with a product model number, then record the product model number of the product image; if no product model number is detected, then perform category recognition on the jewelry image based on a preset multimodal large language model to determine the corresponding jewelry category information; Step S102: Encode the cropped image according to the set domain image vector embedding model to determine the corresponding image vector, input the image vector into the vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on the pre-trained multimodal image vector basic model based on the set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. Step S103: Based on the jewelry category information, filter the candidate product set by category, and deduplicate the filtered results based on the product module number. Re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, and output the preset number of jewelry products after sorting to determine the corresponding jewelry search results.

[0137] As described above, the computer-readable storage medium provided in this application improves input quality by performing target detection and cropping preprocessing on user-uploaded jewelry images, and then performing similar image retrieval and high-definition image replacement through a digital asset management system. Next, based on the similarity retrieval results of the original image, it determines whether the retrieval scope can be limited using product templates. If not, it intelligently identifies jewelry categories using a multimodal large language model, and uses a multimodal vector model fine-tuned with jewelry domain samples to perform feature encoding on the cropped image, generating domain-adaptive image vectors. Based on these vector vectors, it performs similarity recall in the vector database, sequentially performing jewelry category filtering, template deduplication, and multi-dimensional re-sorting that integrates visual similarity, image quality, and product attributes to output the final retrieval results. This improves the efficiency and accuracy of jewelry image retrieval.

[0138] Embodiments of this application also provide a computer program product capable of implementing all steps of the adaptive jewelry image data processing method with the execution subject being a server or client in the above embodiments. When executed by a processor, this computer program / instruction implements the steps of the adaptive jewelry image data processing method. For example, the computer program / instruction implements the following steps: Step S101: Obtain the jewelry image uploaded by the user, perform target detection and target cropping on the main jewelry area of ​​the jewelry image, determine the corresponding cropped image containing only the target jewelry, input the jewelry image into a preset product image library for similarity retrieval, if there is a product image with similarity higher than a preset threshold and the product image is associated with a product model number, then record the product model number of the product image; if no product model number is detected, then perform category recognition on the jewelry image based on a preset multimodal large language model to determine the corresponding jewelry category information; Step S102: Encode the cropped image according to the set domain image vector embedding model to determine the corresponding image vector, input the image vector into the vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on the pre-trained multimodal image vector basic model based on the set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. Step S103: Based on the jewelry category information, filter the candidate product set by category, and deduplicate the filtered results based on the product module number. Re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, and output the preset number of jewelry products after sorting to determine the corresponding jewelry search results.

[0139] As described above, the computer program product provided in this application improves input quality by performing target detection and cropping preprocessing on user-uploaded jewelry images, and then performing similar image retrieval and high-definition image replacement through a digital asset management system. Next, based on the similarity retrieval results of the original image, it determines whether the search scope can be limited using product templates. If not, it intelligently identifies jewelry categories using a multimodal large language model, and uses a multimodal vector model fine-tuned with jewelry domain samples to perform feature encoding on the cropped image, generating domain-adaptive image vectors. Based on these vector vectors, it performs similarity recall in the vector database, sequentially performing jewelry category filtering, template deduplication, and multi-dimensional re-sorting based on visual similarity, image quality, and product attributes, outputting the final search results. This improves the efficiency and accuracy of jewelry image retrieval.

[0140] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. An adaptive jewelry image data processing method, characterized in that, The method includes: The system acquires jewelry images uploaded by users, performs target detection and cropping on the main jewelry area of ​​the images, determines the cropped image containing only the target jewelry, inputs the jewelry images into a preset product image library for similarity retrieval, if there are product images with similarity higher than a preset threshold and the product images are associated with a product model number, then the product model number of the product image is recorded; if no product model number is detected, then the system performs category recognition on the jewelry images based on a preset multimodal large language model to determine the corresponding jewelry category information. The cropped image is encoded according to a defined domain image vector embedding model to determine the corresponding image vector. The image vector is then input into a vector database for similarity retrieval to determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on a pre-trained multimodal image vector basic model based on a defined jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. Based on the jewelry category information, the candidate product set is filtered by category, and the filtered results are deduplicated based on the product module number. The deduplicated candidate product set is then reordered according to image vector similarity, image quality index, and product attribute matching degree. A preset number of jewelry products are then output to determine the corresponding jewelry search results.

2. The adaptive jewelry image data processing method according to claim 1, characterized in that, Before inputting the jewelry image into a preset product image library for similarity retrieval, the process includes: The jewelry image is input into a preset high-definition asset database for similarity retrieval. If an asset image that meets the criteria is found, the jewelry image is replaced with the asset image, and a subsequent similarity retrieval is performed based on the replaced jewelry image. The asset image that meets the criteria includes an asset image whose similarity to the original jewelry image is higher than a preset threshold and whose quality meets the preset high-definition criteria. The replaced jewelry image is subjected to target detection and target cropping to determine the corresponding new cropped image containing the target jewelry. The original cropped image is replaced with the new cropped image, and subsequent image vector encoding is performed based on the replaced cropped image.

3. The adaptive jewelry image data processing method according to claim 1, characterized in that, The step of classifying the jewelry image based on a preset multimodal large language model to determine the corresponding jewelry category information includes: The jewelry image is obtained as input, and cross-modal feature extraction is performed on the input image according to a preset multimodal large language model to determine the structured semantic representation corresponding to the image. The multimodal large language model is obtained by pre-training on a natural image-text pairing dataset. The classification module performs category discrimination on the structured semantic representation according to the set classification module to determine the corresponding jewelry category information. The classification module is obtained after adaptive fine-tuning training for the jewelry field. The fine-tuning training is based on the jewelry dataset, which includes at least one of wearing scene images, tiled images, and local detail images, and the annotation information covers at least one of bracelets, necklaces, pendants, rings, and earrings.

4. The adaptive jewelry image data processing method according to claim 1, characterized in that, The step of performing domain-adaptive fine-tuning training on the pre-trained multimodal image vector base model based on a set jewelry domain sample dataset includes: Construct a sample dataset for the jewelry domain, including: Collect jewelry images containing product templates as raw data, and automatically construct positive sample pairs based on the product templates. The positive sample pairs are multiple images of the same template from different angles or in different scenes. Negative sample pairs are constructed based on different product templates, wherein the negative sample pairs are combinations of images with different randomly selected templates; Sample pairs with similar appearances but different product models are introduced as interference samples to enhance the model's ability to identify fine-grained differences; The jewelry domain sample dataset is input into a pre-trained multimodal image vector base model. A loss function is constructed based on the sample pair label information. The loss function includes a positive sample loss term for constraining the vector distance between positive sample pairs and a negative sample loss term for constraining the vector distance between negative sample pairs. The parameters of the pre-trained multimodal image vector base model are iteratively optimized using the backpropagation algorithm until the loss function converges, thereby determining the corresponding domain image vector embedding model.

5. The adaptive jewelry image data processing method according to claim 1, characterized in that, The step of inputting the image vector into a vector database for similarity retrieval to determine the corresponding candidate product set includes: Detect whether the image vector is associated with a recorded product template; If a recorded product template exists, the vector database is pre-indexed based on the product template to determine the corresponding pre-screened subset after template filtering, and the image vector is searched for similarity based on the pre-screened subset to determine the corresponding candidate product set. If no recorded product template number exists, the image vector is input into the vector database, and a large-scale data retrieval is performed within the vector database to conduct a similarity search on the image vector and determine the corresponding candidate product set.

6. The adaptive jewelry image data processing method according to claim 1, characterized in that, The step of filtering the candidate product set based on the jewelry category information and deduplicating the filtered results based on the product module number includes: Obtain the product attribute information of each candidate product in the candidate product set, match the product attribute information with the jewelry category information, retain the candidate products that match successfully, and remove the candidate products that do not match successfully. For the candidate products retained after matching, their corresponding product template numbers are extracted. For multiple candidate products with the same product template number, one candidate product is selected as the representative result of the product template number according to the preset deduplication rule.

7. The adaptive jewelry image data processing method according to claim 1, characterized in that, The process of re-sorting the deduplicated candidate product set based on image vector similarity, image quality indicators, and product attribute matching degree, and outputting a preset number of jewelry products after sorting, determines the corresponding jewelry search results, including: For the deduplicated candidate product set, image quality is evaluated using a preset image quality evaluation model to determine the corresponding product image quality index. Based on the product image quality index, combined with image vector similarity and product attribute matching degree, the product images in the candidate product set are re-ranked, and the product images with the highest ranking are output according to a preset number to determine the corresponding jewelry search results.

8. An adaptive jewelry image data processing device, characterized in that, The device includes: The jewelry image processing module is used to acquire jewelry images uploaded by users, perform target detection and target cropping on the main jewelry area of ​​the jewelry images, determine the corresponding cropped image containing only the target jewelry, input the jewelry images into a preset product image library for similarity retrieval, if there are product images with similarity higher than a preset threshold and the product images are associated with product model numbers, then the product model number of the product images is recorded; if no product model number is detected, then the jewelry images are classified based on a preset multimodal large language model to determine the corresponding jewelry category information. The jewelry product set determination module is used to encode the cropped image according to a set domain image vector embedding model, determine the corresponding image vector, input the image vector into a vector database for similarity retrieval, and determine the corresponding candidate product set. The domain image vector embedding model is obtained by performing domain adaptive fine-tuning training on a pre-trained multimodal image vector basic model based on a set jewelry domain sample dataset. The sample dataset includes positive image sample pairs of the same jewelry product under different shooting angles and wearing scenarios, negative image sample pairs of different jewelry products, and interference sample pairs that are similar in appearance but different in model number. The jewelry search result determination module is used to filter the candidate product set based on the jewelry category information, remove duplicates from the filtered results based on the product module number, re-sort the deduplicated candidate product set according to image vector similarity, image quality index and product attribute matching degree, output a preset number of sorted jewelry products, and determine the corresponding jewelry search results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the adaptive jewelry image data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the adaptive jewelry image data processing method according to any one of claims 1 to 7.

Images