Same-model matching method and device for commodity image and storage medium

Abstract

The application discloses a same-model matching method and device for commodity images and a storage medium, and relates to the field of computer vision. The method comprises the following steps: acquiring a current commodity image set; extracting a corresponding global feature vector and a local feature vector for each commodity image in the current commodity image set, fusing the global feature vector and the local feature vector, and obtaining a comprehensive feature vector corresponding to each commodity image; grouping each commodity image according to a commodity identification rule, clustering and bucketing the grouping result according to a preset number, and obtaining a plurality of image buckets; and performing image similarity matching based on the comprehensive feature vector in each image bucket and between each image bucket, and obtaining a same-model image combination. The application can fuse the global and local features of commodities to form a comprehensive feature vector with better representation, and adopts a hierarchical matching mechanism of grouping first and then bucketing, so that the overall processing efficiency is improved while the same-model matching precision is ensured.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, device and storage medium for matching product images. Background Technology

[0002] As the scale of goods on e-commerce and content platforms continues to expand, the number of new product images added daily across various channels is enormous. To improve search accuracy and advertising conversion rates, platforms need to perform same-item identification and aggregation matching of product images during the data entry stage.

[0003] Currently, existing methods for matching similar products generally employ a single-feature model to vectorize product images and then perform a full pairwise comparison, using a similarity threshold to determine whether an image is classified as a similar product. However, because a single feature cannot fully capture the diverse information contained in a product image, it is prone to misjudgment when faced with a large number of product images with complex backgrounds, similar styles, but different details, thus affecting the accuracy of similar product identification. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device and storage medium for matching product images, aiming to solve the technical problem that the accuracy of matching product images is low due to existing single-feature-based matching methods.

[0005] To achieve the above objectives, this application proposes a method for matching similar products in product images, the method comprising:

[0006] Get the current collection of product images;

[0007] For each product image in the current product image set, extract the corresponding global feature vector and local feature vector, and fuse the global feature vector and local feature vector to obtain the comprehensive feature vector corresponding to each product image.

[0008] The product images are grouped according to the product identification rules, and the grouping results are clustered and bucketed according to a preset number to obtain several image buckets;

[0009] Image similarity matching is performed within each image bucket and between each image bucket based on the comprehensive feature vector, and a combination of images with the same style is obtained based on the matching results.

[0010] In one embodiment, the step of extracting the corresponding global feature vector and local feature vector for each product image in the current product image set, and fusing the global feature vector and local feature vector to obtain the comprehensive feature vector corresponding to each product image includes:

[0011] Each product image in the current product image set is input into a preset visual feature extraction model to obtain a global feature vector corresponding to each product image.

[0012] Each product image in the current product image set is input into a preset segmentation model to obtain a segmentation mask corresponding to each product image, and a local feature vector corresponding to each product image is generated based on each segmentation mask.

[0013] The global feature vector and the local feature vector are fused according to preset weights to obtain the comprehensive feature vector corresponding to each product image.

[0014] In one embodiment, the step of inputting each product image in the current product image set into a preset visual feature extraction model to obtain a global feature vector corresponding to each product image includes:

[0015] Each product image in the current product image set is input into a preset visual feature extraction model to obtain the visual features corresponding to each product image.

[0016] The visual features are normalized to obtain the global feature vector corresponding to each product image;

[0017] The step of inputting each product image in the current product image set into a preset segmentation model to obtain a segmentation mask corresponding to each product image, and generating a local feature vector corresponding to each product image based on each segmentation mask, includes:

[0018] Each product image in the current product image set is input into a preset segmentation model to obtain the segmentation mask corresponding to each product image;

[0019] Extract the mask contour of each segmentation mask, determine the chord angle histogram vector corresponding to each product image based on each mask contour, and normalize each chord angle histogram vector to obtain the local feature vector corresponding to each product image.

[0020] In one embodiment, the step of grouping the product images according to product identification rules and clustering the grouping results into buckets according to a preset number to obtain several image buckets includes:

[0021] The product images are grouped according to their grouping identifiers to obtain several image groups, and the product images in each image group have the same grouping identifier.

[0022] In each of the image groups, the product images are clustered according to the product identifier of each product image, and several simplified image groups are determined based on the clustering results. The number of product images with the same product identifier in each simplified image group does not exceed a preset number.

[0023] When the number of product identifiers in any of the simplified image groups reaches a preset threshold, a dynamic bucketing strategy is executed on each of the simplified image groups to obtain several image buckets.

[0024] In one embodiment, the step of performing image similarity matching based on the comprehensive feature vector within and between each of the image buckets, and obtaining a combination of images with the same style based on the matching results, includes:

[0025] In each of the image buckets, similarity matching is performed based on the comprehensive feature vector corresponding to the product image to obtain the bucket matching result, and the optimal image combination within the bucket is determined based on the bucket matching result.

[0026] The representative image corresponding to each image bucket is determined based on the optimal image combination within the bucket.

[0027] Similarity matching is performed between the image buckets based on the comprehensive feature vectors corresponding to the representative images to obtain the bucket matching results;

[0028] Based on the inter-bucket matching results, high-matching image buckets are determined, and similarity matching is performed based on the comprehensive feature vectors corresponding to the product images in the high-matching image buckets to obtain extended matching results;

[0029] The optimal image combination is updated based on the extended matching results to obtain the same image combination.

[0030] In one embodiment, the step of performing similarity matching based on the comprehensive feature vector corresponding to the product image in each of the image buckets to obtain the intra-bucket matching result, and determining the optimal image combination within the bucket based on the intra-bucket matching result, includes:

[0031] Based on the image binning, the similarity between the comprehensive feature vectors corresponding to the product images is determined, and a similarity matrix is ​​obtained;

[0032] Based on the similarity matrix, several similar image combinations are determined in descending order of similarity, and several similar image combinations that rank higher than a preset number are selected as candidate image combinations.

[0033] Conflict detection is performed on each candidate image combination, and candidate image combinations with conflicts are eliminated to obtain the optimal image combination within the bucket.

[0034] In one embodiment, the step of updating the optimal image combination based on the extended matching result to obtain the same image combination includes:

[0035] The optimal image combination is updated based on the extended matching results to obtain the updated optimal image combination.

[0036] The updated optimal image combination is then validated a second time according to the preset product parameter dimensions, and the same image combination is obtained based on the validation results.

[0037] In one embodiment, the step of obtaining the current set of product images includes:

[0038] Obtain the original set of product images, and sort the original set of product images according to the preset product category filtering rules to obtain the effective set of product images;

[0039] The valid set of product images is subjected to image deduplication to obtain the current set of product images.

[0040] Furthermore, to achieve the above objectives, this application also proposes a product image matching device, the device comprising: a memory, a processor, and a product image matching program stored in the memory and executable on the processor, the product image matching program being configured to implement the steps of the product image matching method described above.

[0041] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, storing a matching program for product images. When the matching program for product images is executed by a processor, it implements the steps of the matching method for product images as described above.

[0042] This application discloses a method for matching similar products in product images. The method includes: obtaining a current set of product images; extracting the corresponding global feature vector and local feature vector for each product image in the current set of product images, fusing the global feature vector and local feature vector to obtain a comprehensive feature vector corresponding to each product image; grouping each product image according to product identification rules, and clustering the grouping results into buckets according to a preset number to obtain several image buckets; performing image similarity matching based on the comprehensive feature vector within each image bucket and between each image bucket, and obtaining a combination of similar images based on the matching results.

[0043] This application enhances the completeness and discriminative power of product image descriptions by fusing global and local features to form a more representative comprehensive feature vector, thereby improving the accuracy and distinctiveness of product image descriptions at the feature level. This enables more precise capture of the core visual characteristics of similar products during subsequent matching. Furthermore, by employing a hierarchical processing structure of grouping followed by binning, the large-scale image matching problem is decomposed into multiple smaller problems that can be processed in parallel, thus significantly improving overall matching efficiency while maintaining matching accuracy. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the first embodiment of the product image matching method of this application;

[0047] Figure 2 This is a schematic diagram of the dual-feature fusion process of the product image in this application;

[0048] Figure 3 This is a flowchart illustrating the second embodiment of the product image matching method of this application;

[0049] Figure 4 This is a logical diagram illustrating the grouping and binning mechanism used in this application to divide product images;

[0050] Figure 5 This is a flowchart illustrating the third embodiment of the product image matching method of this application;

[0051] Figure 6 This is a schematic diagram of the structure of the matching device for the product image in the embodiments of this application.

[0052] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0054] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0055] This application provides a method for matching product images to the same style, referencing... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the product image matching method of this application. In this embodiment, the method includes: steps S10~S40:

[0056] Step S10: Obtain the current product image set.

[0057] It should be noted that the method of this embodiment can be applied to scenarios of automatic deduplication, similarity recognition, and optimal display of product images on e-commerce platforms. The executing entity of this embodiment can be a computing electronic device with functions such as data processing, network communication, and program execution, such as a desktop computer, a mainframe computer, or a server. Here, a product image similarity matching device (referred to as "device") is used as an example to illustrate this embodiment and the following embodiments.

[0058] It should be understood that the current set of product images can be obtained by deduplicating the original product image data. The original product image data, i.e., the original set of product images, can include all product images from data sources (e-commerce platform product databases, uploaded advertising material libraries, etc.) that need to be identified and matched.

[0059] After obtaining the original set of product images, differentiated filtering rules can be formulated based on the characteristics of product categories to selectively filter the input original set of product images, removing images that do not meet the display and matching requirements (such as detail images, back images, close-up images, etc.), and retaining effective images such as front images and main images that can fully represent the characteristics of the product, thus obtaining an effective set of product images.

[0060] Next, the image deduplication interface can be called to perform preliminary redundancy filtering on the valid product image set, removing completely duplicate images from the set (such as images of the same product uploaded repeatedly at the same angle and resolution), further reducing the amount of data, retaining the valid sample set, and finally obtaining the current product image set.

[0061] In practice, the device can first acquire the original set of product images, then sort the original set of product images according to the preset product category filtering rules to obtain the effective set of product images; finally, it can perform image deduplication on the effective set of product images to obtain the current set of product images.

[0062] Step S20: Extract the corresponding global feature vector and local feature vector for each product image in the current product image set, and fuse the global feature vector and local feature vector to obtain the comprehensive feature vector corresponding to each product image.

[0063] It should be noted that the global feature vector can be the visual features of the product image, which can be a high-dimensional vector used to represent the overall content of the product image (such as color, texture, layout, style); the local features can be the shape features of the product image, which can be a low-dimensional feature vector used to represent the outline of the product image.

[0064] Furthermore, this can also be combined with Figure 2 The process of extracting dual features (global features and local features) from product images is explained. Figure 2 This is a schematic diagram of the dual-feature fusion process of the product image in this application. For specific illustration, step S20 includes: steps S201~S203:

[0065] Step S201: Input each product image in the current product image set into the preset visual feature extraction model to obtain the global feature vector corresponding to each product image.

[0066] It should be noted that the preset visual feature extraction model can be a pre-trained model for global semantic feature extraction of product images, such as the multimodal image-text model SigLIP, the neural network model CNN, etc. The following explanation will use the SigLIP model as an example.

[0067] Before inputting the product image into the SigLIP model, the product image can be resized and its pixel values ​​normalized, so that the resized product image meets the input requirements of the preset SigLIP model.

[0068] The SigLIP model supports FP16 precision inference and can be deployed on devices via the Unified Computing Device Architecture (CUDA), thereby leveraging the parallel computing capabilities of the device's GPU at runtime to efficiently extract semantic features from product images.

[0069] After obtaining the original feature vectors input to the SigLIP model, L2 normalization can be performed on them to unify the magnitude of the feature vectors, eliminate the differences in dimensions and scales in the feature dimensions, and finally obtain a high-dimensional (e.g., 1024-dimensional) global feature vector that represents the global content and semantics of the image.

[0070] In a specific implementation, the device can input each product image in the current product image set into a preset visual feature extraction model to obtain the visual features corresponding to each product image; then, it can perform normalization processing on each visual feature to obtain the global feature vector corresponding to each product image.

[0071] Step S202: Input each product image in the current product image set into the preset segmentation model to obtain the segmentation mask corresponding to each product image, and generate the local feature vector corresponding to each product image based on the segmentation mask.

[0072] It should be understood that this preset segmentation model can be a preset image segmentation API (in...) Figure 2 The API is denoted as MATTING_API. It can also be a pre-trained model for image contour segmentation, such as Mask R-CNN. This embodiment does not limit this, and the image segmentation API is used as an example for explanation.

[0073] When an image segmentation API is called to process the input product image, the API can return a segmentation mask for the product image. This segmentation mask can be a binary image of the same size as the input product image, where white areas represent the product and black areas represent the background, thus accurately separating the product target.

[0074] It should be noted that an image processing library (cv2.findContours) can also be used to extract the mask contour of the product target from the segmentation mask, and select the contour with the largest area as the main contour line of the product target; the mask contour is centered and scaled to be within a unit circle, thereby eliminating the influence of the position and size of the product target in the product image, and making the shape features translational and scale invariant; feature point sampling and feature normalization are performed on the mask contour to obtain a fixed number (e.g., 128) of feature points, i.e., contour points, thereby reducing the amount of data.

[0075] Next, based on the sampled contour points, their chord angle histogram can be calculated. Specifically, this can be achieved by statistically analyzing the distribution of chord angles formed by adjacent points on the contour points (…). Figure 2 In the example shown, the data is divided into 36 bins for statistical analysis, quantizing the outline shape of the product target into a low-dimensional, discriminative feature vector. This ultimately yields a low-dimensional (e.g., 36-dimensional) local feature vector representing the product's shape and structure.

[0076] In a specific implementation, the device can input each product image in the current product image set into a preset segmentation model to obtain the segmentation mask corresponding to each product image; extract the mask contour of each segmentation mask; determine the chord angle histogram vector corresponding to each product image based on each mask contour; and normalize each chord angle histogram vector to obtain the local feature vector corresponding to each product image.

[0077] Step S203: The global feature vector and the local feature vector are fused according to preset weights to obtain the comprehensive feature vector corresponding to each product image.

[0078] It should be noted that this preset weight can be a pre-set weight factor. It can dynamically adjust based on the product category characteristics corresponding to the product image set (e.g., the visual feature weight can be appropriately increased for clothing, and the shape feature weight can be appropriately increased for footwear). For example, if... If the sum is 0.5, then the comprehensive eigenvector can be expressed as: Comprehensive eigenvector = 0.5 Global feature vector +0.5 Local feature vector. This comprehensive feature vector can serve as a unique representative of the product image and is used for subsequent matching calculations.

[0079] Furthermore, for example, in the clothing category, one can... Adjust to 0.6 to enhance the impact of visual features (color, texture); in the footwear category, this can be... Adjusted to 0.4 to enhance the influence of silhouette features (shoe shape, sole structure).

[0080] Step S30: Group each product image according to the product identification rules, and cluster the grouping results into buckets according to a preset number to obtain several image buckets.

[0081] It should be noted that the product identification rules can be the product's brand identifier (group_name) and the product identifier (goods_id).

[0082] It should be understood that product images can first be grouped according to brand identifier (group_name) to ensure that product images in the same group belong to the same target (i.e., the same series of products under the same brand), thereby avoiding invalid matching across groups.

[0083] Next, within each image group, clusters can be formed based on the product's identifier (goods_id), with each cluster corresponding to a single goods_id. Each cluster retains a maximum of a fixed number (e.g., 3) of representative product images, thereby further reducing the amount of image data within the image group.

[0084] Furthermore, determine whether the number of items within a single image group has reached a preset quantity (e.g., whether...). If the target number of items is 8, a dynamic bucketing strategy can be initiated: the items within the image group are divided into multiple image buckets. The capacity of each bucket can be adaptively adjusted based on the total number of items. For example, the default bucket capacity is 5, and when the total number of items is 12, it is divided into 3 buckets with capacities of 4, 4, and 4 respectively. This ensures that the computational complexity within each bucket is controllable and avoids a decrease in processing efficiency due to excessive data volume in a single bucket.

[0085] Step S40: Perform image similarity matching based on the comprehensive feature vector within each image bucket and between each image bucket, and obtain a combination of images with the same style based on the matching results.

[0086] It should be noted that, firstly, within each independent image bucket, the similarity between all product images can be calculated based on the comprehensive feature vector, and then the same product within the bucket can be quickly identified through optimization algorithms.

[0087] Next, after completing all intra-bucket matching, representative images from each image bucket (such as the top two images with the highest similarity within a bucket) can be selected for cross-bucket matching. If a high similarity is detected between representative images across buckets (e.g., similarity...), the matching will proceed across buckets. If the result is 0.9, then the matching will be extended to all product images in the corresponding bucket for a second, more precise match, ensuring that no identical products across buckets are missed.

[0088] Finally, the results of intra-bucket matching and the corrected results of cross-bucket matching can be merged to output the final list of identical image combinations.

[0089] This embodiment can form a more representative comprehensive feature vector by fusing global and local features of products, thereby improving the completeness and distinguishability of product image description at the feature level. This enables more accurate capture of the core visual characteristics of similar products during subsequent matching. Furthermore, by using a hierarchical processing structure of grouping and then binning, the large-scale image matching problem is decomposed into multiple smaller problems that can be processed in parallel, thus significantly improving the overall matching efficiency while ensuring matching accuracy.

[0090] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the product image matching method of this application.

[0091] In this embodiment, to specifically illustrate how the grouping and binning mechanism is used to divide the product images, step S30 includes: steps S301~S303:

[0092] Step S301: Group the product images according to their grouping identifiers to obtain several image groups, wherein the product images in each image group have the same grouping identifier.

[0093] It should be noted that the group identifier for product images is the brand identifier (group_name). There can be multiple different products under the same brand identifier (group_name).

[0094] In the implementation, all product images are categorized according to brand identifier (group_name) to obtain several image groups. Each image group corresponds to a brand identifier (group_name), and all product images within the same image group have the same brand identifier (group_name), thus avoiding subsequent cross-brand matching and reducing unnecessary computation.

[0095] For example, all product images of "Brand A Series Sneakers" can be grouped into the same image group, while all product images of "Brand B Series T-shirts" can be grouped into another image group.

[0096] Step S302: In each of the image groups, the product images are clustered according to the product identifier of each product image, and several simplified image groups are determined according to the clustering results. The number of product images with the same product identifier in each simplified image group does not exceed a preset number.

[0097] It should be noted that within the image groups obtained by categorizing by brand identifier, considering that there can be multiple different products under the same brand identifier (group_name), and that each product category identifier (goods_id) uniquely corresponds to one product, the same product (same product identifier) ​​can have multiple product images. For example, for sneaker A with goods_id=12345, it can have multiple product photos from different angles and in different scenes.

[0098] Therefore, clustering can be performed based on the product identifier (goods_id) of the product images. Representative product images (such as front view, side view, and close-up detail images) can be selected from multiple product images of the same product, and highly repetitive product images can be removed to obtain simplified image groups. In this simplified image group, the number of product images under the same product identifier (goods_id) does not exceed a preset number (which can be set to 3).

[0099] Step S303: When the number of product identifiers in any of the simplified image groups reaches a preset threshold, a dynamic bucketing strategy is executed on each of the simplified image groups to obtain several image buckets.

[0100] It should be understood that after obtaining each simplified image group, the number of different products contained in each simplified image group, i.e., the number of unique goods_ids, can be determined and denoted as N; then, it is determined whether the number of products in each simplified image group has reached a preset threshold. This preset threshold is used to determine the number of products under the same brand, for example, it can be set to 8.

[0101] If the number of goods, i.e., the number of goods identifiers (goods_id) in a certain simplified image group is not less than the above threshold, then it is determined that the simplified image group needs to implement a dynamic bucketing strategy. The dynamic bucketing strategy can be represented as follows:

[0102] If N < 8, it is considered that the number of products in the group is small and the computational complexity is controllable. Therefore, no binning is performed and the entire group can be directly regarded as an image bin.

[0103] If N 8: Then, based on the total number of goods N in the group and a preset bucket capacity M (e.g., M=5), the required number of buckets K = ceil(N / M) (i.e., rounded up) can be automatically calculated.

[0104] In addition, when performing image binning, a "pre-clustering by feature similarity" strategy can be adopted. First, the similarity of the comprehensive features of the products is calculated, and products with high similarity of comprehensive features are assigned to the same bin, thereby reducing the amount of computation for cross-bin matching.

[0105] It should also be noted that, unlike the comprehensive feature vectors corresponding to individual product images, the comprehensive product vector can be the one corresponding to the product identifier (goods_id). That is, the comprehensive product vector can be the average of the comprehensive feature vectors of different product images under the same product identifier.

[0106] In the specific implementation, the above dynamic bucketing strategy is applied to each simplified image group to obtain several image buckets, including image buckets for small-scale scenarios and image buckets for large-scale scenarios. Image buckets for small-scale scenarios are simplified image groups with fewer than a preset threshold of products within the group, while image buckets for large-scale scenarios are obtained by further dividing simplified image groups with more than or equal to the preset threshold of products within the group. This ensures that the computational complexity within each bucket is controllable and avoids a decrease in processing efficiency due to excessively large data volume in a single bucket.

[0107] Furthermore, this can be referenced here. Figure 4 The above process of grouping and separating items into buckets will be explained. Figure 4 This is a logical diagram illustrating the grouping and binning mechanism used in this application to divide product images.

[0108] First, the product images can be grouped according to their brand identifier (group_name) to obtain several image groups corresponding to the brand identifier (group_name);

[0109] Next, within each image group, the images are further divided according to the product category identifier (goods_id) and multi-image clustering of the products is performed to obtain the simplified image groups;

[0110] Determine whether the total number of goods in the simplified image group, i.e., the number of goods_id, has reached the preset threshold.

[0111] If the number of goods_ids reaches the preset threshold, the group is determined to correspond to a large-scale scenario. Based on the total number of goods N in the group and a preset bucket capacity M (e.g., M=5), the required number of buckets K is automatically calculated to obtain several image buckets.

[0112] After obtaining each image bucket, image clustering within the bucket can be further implemented to ensure that the number of product images under the same product identifier (goods_id) in each image bucket does not exceed the preset number (which can be set to 3 images).

[0113] If the number of goods_ids has not reached the preset threshold, the simplified image group will be directly regarded as an image bucket, and the number of product images under the same product identifier (goods_id) in the image bucket will not exceed the preset number (which can be set to 3).

[0114] This embodiment groups product images according to grouping identifiers, then clusters within each image group based on the product identifiers to reduce the number of images in the group, and finally adopts a dynamic bucketing strategy based on the number of products in a single reduced image group, thereby ensuring that the size of each final image bucket is small enough to facilitate efficient intra-bucket matching in the future.

[0115] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the product image matching method of this application.

[0116] In this embodiment, to specifically illustrate how to perform matching based on image buckets after obtaining each image bucket, step S40 further includes: steps S401~S405:

[0117] S401: In each of the image buckets, similarity matching is performed based on the comprehensive feature vector corresponding to the product image to obtain the bucket matching result, and the optimal image combination within the bucket is determined based on the bucket matching result.

[0118] It should be understood that in each image bucket, the similarity of the product images in the bucket can first be compared (usually using cosine similarity) to form a similarity matrix. Then, a combination optimization algorithm (heap sort combined with conflict detection) can be used to select the most similar image pairs or groups from the above similarity matrix.

[0119] For example, for an image bucket, after similarity matching, the optimal combination of images within the bucket can be: combination 1 {image A, image B} and combination 2 {image C, image D}.

[0120] Furthermore, to illustrate in detail how to implement the combinatorial optimization algorithm, step S401 also includes: steps S4011~S4012:

[0121] Step S4011: Based on each image bucket, determine the similarity between the comprehensive feature vectors corresponding to the product images to obtain a similarity matrix.

[0122] It should be noted that for any image bin, the pairwise similarity between all product images within the bin can be calculated, that is, the cosine similarity is calculated using the comprehensive feature vector of the product images.

[0123] The similarity matrix can be a symmetric matrix, where each element s(i,j) represents the cosine similarity between the i-th product image and the j-th product image within the aforementioned image bucket.

[0124] Step S4012: Based on the similarity matrix, determine a number of similar image combinations in descending order of similarity, and select a number of similar image combinations that are ranked before a preset number as candidate image combinations.

[0125] It should be noted that the device can traverse the above similarity matrix, extract and process all non-repeating image pairs (such as image AB, image AC, etc.), and sort them from high to low according to their corresponding cosine similarity values.

[0126] Next, the top-ranked image pairs can be selected from the above sorting as "candidate image combinations". The preset ranking can be set to a specific number, such as the top 20 pairs; it can also be set based on a similarity threshold, such as image pairs with a cosine similarity higher than 0.85. This embodiment does not limit this.

[0127] Step S4013: Perform conflict detection on each candidate image combination and remove candidate image combinations that have conflicts to obtain the optimal image combination in the bucket.

[0128] It should be noted that conflict detection can detect whether there is a conflict in business logic between two product images in a candidate image combination, such as whether there is a conflict in product attributes.

[0129] If two product images belong to the same candidate image set, but their product attributes are displayed as different colors, different sizes, or different materials, then a conflict is constituted.

[0130] In the specific implementation, the above-mentioned conflict detection is performed on each candidate image combination, and the candidate image combinations with conflicts are eliminated. The remaining candidate image combinations can then be determined as the optimal image combinations within the bucket.

[0131] S402: Determine the representative image corresponding to each image bucket based on the optimal image combination within the bucket.

[0132] It should be noted that after completing the intra-bucket matching, a representative image can be selected for each image by dividing it into buckets. This representative image can be the two images with the highest similarity in the bucket, or it can be selected from the optimal combination of images in the bucket.

[0133] For example, if the optimal image combination within a certain image bucket is: combination 1 {image A, image B} and combination 2 {image C, image D}; then in combination 1, image A can be selected; in combination 2, image C can be selected; then the representative images of this image bucket can be image A and image C.

[0134] S403: Similarity matching is performed between each of the image buckets based on the comprehensive feature vectors corresponding to each representative image to obtain the bucket matching result.

[0135] S404: Determine the high-matching image buckets based on the inter-bucket matching results, and perform similarity matching based on the comprehensive feature vectors corresponding to the product images in the high-matching image buckets to obtain extended matching results.

[0136] It should be noted that the device can perform pairwise matching among all representative images, calculating the similarity between the comprehensive feature vectors of different representative images. A similarity threshold between buckets can be set, for example, to 0.9. Therefore, if the similarity between the representative images of two image buckets exceeds 0.9, then those two image buckets can be identified as high-matching image buckets.

[0137] It should be understood that after determining the high-matching image buckets, full similarity matching can then be performed on all product images in the high-matching image buckets to obtain extended matching results. These extended matching results can include the optimal image combinations across buckets, such as combination 3 {image A, image E} and combination 2 {image B, image F}.

[0138] S405: Update the optimal image combination based on the extended matching result to obtain the same image combination.

[0139] It should be noted that the device can update the aforementioned optimal image combination based on the extended matching results. For example, it can extend the aforementioned optimal image combination to obtain a global and accurate combination of the same image.

[0140] For example, if the optimal image combination within a certain image bucket is: combination 1 {image A, image B} and combination 2 {image C, image D}; and the extended matching result obtains a new optimal image combination across buckets: combination 3 {image A, image E} and combination 2 {image B, image F}; then the updated optimal image combination can be {image A, image B, image E, image F}.

[0141] In addition, to further ensure the accuracy of the output image combination, after obtaining the updated optimal image combination, a second verification can be performed on the updated optimal image combination according to the preset product parameter dimensions, and the image combination can be obtained based on the verification result.

[0142] It should be understood that the preset product parameter dimensions can be other product business attributes besides visual ones, such as price, size, color, material, etc.

[0143] For example, considering that product prices typically fluctuate within a small range, if two product images identified as the same item have significantly different prices (e.g., exceeding 10%), it may be a false match; if two product images identified as the same item differ in color or material, it may be a false match; if two product images identified as the same item differ in size, since the same item may have multiple sizes, it may not be a false match. Specific rules can be formulated based on differences for different product categories (footwear, clothing, home appliances, etc.), and this embodiment does not impose any limitations on this.

[0144] In a specific implementation, after updating the optimal image combination based on the extended matching result and obtaining the updated optimal image combination device, the device can perform a secondary verification on each combination in the updated final image combination to determine whether all product images in the combination are consistent in the preset parameter dimensions or conform to the preset compatibility rules, and then delete the mismatched product images from the updated final image combination, and finally obtain the same image combination.

[0145] This embodiment significantly reduces the initial computational threshold for cross-bucket matching by using representative images, reducing the computational complexity of large-scale product images from the traditional O(n) to O(n)^2. 2 The computation time was reduced to O(n), where n is the number of product images. A second, precise check was performed using extended matching to ensure the accuracy of the final result, avoiding misjudgments and omissions caused by binning. This ensured that even if the same product was divided into different bins, it could still be correctly identified and grouped together. Furthermore, a second check based on product attributes was performed on the obtained optimal image combination, further improving the accuracy of the final output of the same product image combination.

[0146] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the matching method of product images in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0147] This application also provides a product image matching device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the product image matching method in Embodiment 1 above.

[0148] The following is for reference. Figure 6 , Figure 6 This is a schematic diagram of the structure of a matching device for the product images in this application. The matching device for the product images in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), etc., as well as fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The matching device shown for the product image is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0149] like Figure 6As shown, the product image matching device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the product image matching device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the matching device for the product image to communicate wirelessly or wiredly with other devices to exchange data. Although matching devices for the product image with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0150] The product image matching device provided in this application, employing the product image matching method described in the above embodiments, can solve the technical problem of matching product images to the same style. Compared with the prior art, the beneficial effects of the product image matching device provided in this application are the same as those of the product image matching method provided in the above embodiments, and other technical features in this product image matching device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0151] This application also provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the product image matching method in the above embodiments.

[0152] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0153] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described product image matching method, thereby solving the technical problem of the product image matching method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the product image matching method provided in the above embodiments, and will not be repeated here.

[0154] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other elements in the process, method, article, or system that includes that element.

[0155] The sequence numbers of the above embodiments of the present invention are merely for description and do not represent the superiority or inferiority of the embodiments. They are only some embodiments of this application and are not intended to limit the scope of this application. All equivalent structural transformations made under the technical concept of this application and based on the content of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.

Claims

1. A method for matching similar products in product images, characterized in that, The method includes: Get the current collection of product images; For each product image in the current product image set, extract the corresponding global feature vector and local feature vector respectively, and fuse the global feature vector and local feature vector to obtain the comprehensive feature vector corresponding to each product image. The global feature vector is extracted by a preset visual feature extraction model and is used to characterize the visual features of the overall content of the product image. The local feature vector is extracted by a preset segmentation model and is used to characterize the shape features of the product image outline. The product images are grouped according to the product identification rules, and the grouping results are clustered and bucketed according to a preset number to obtain several image buckets; Image similarity matching is performed within each image bucket and between each image bucket based on the comprehensive feature vector, and a combination of images with the same style is obtained based on the matching results; The step of grouping the product images according to product identification rules and clustering the grouping results into buckets according to a preset number to obtain several image buckets includes: The product images are grouped according to their respective grouping identifiers to obtain several image groups. The product images in each image group have the same grouping identifier, which is a brand identifier. In each of the image groups, the product images are clustered according to the product identifier of each product image, and several simplified image groups are determined based on the clustering results. The number of product images with the same product identifier in each simplified image group does not exceed a preset number. When the number of product identifiers in any of the simplified image groups reaches a preset threshold, a dynamic bucketing strategy is executed on each of the simplified image groups to obtain several image buckets. The dynamic bucketing strategy is a strategy that automatically calculates the number of buckets based on the total number of products in the group and the preset bucket capacity.

2. The method as described in claim 1, characterized in that, The step of extracting the corresponding global feature vector and local feature vector for each product image in the current product image set, and fusing the global feature vector and local feature vector to obtain the comprehensive feature vector corresponding to each product image includes: Each product image in the current product image set is input into a preset visual feature extraction model to obtain a global feature vector corresponding to each product image. Each product image in the current product image set is input into a preset segmentation model to obtain a segmentation mask corresponding to each product image, and a local feature vector corresponding to each product image is generated based on each segmentation mask. The global feature vector and the local feature vector are fused according to preset weights to obtain the comprehensive feature vector corresponding to each product image.

3. The method as described in claim 2, characterized in that, The step of inputting each product image in the current product image set into a preset visual feature extraction model to obtain the global feature vector corresponding to each product image includes: Each product image in the current product image set is input into a preset visual feature extraction model to obtain the visual features corresponding to each product image. The visual features are normalized to obtain the global feature vector corresponding to each product image; The step of inputting each product image in the current product image set into a preset segmentation model to obtain a segmentation mask corresponding to each product image, and generating a local feature vector corresponding to each product image based on each segmentation mask, includes: Each product image in the current product image set is input into a preset segmentation model to obtain the segmentation mask corresponding to each product image; Extract the mask contour of each segmentation mask, determine the chord angle histogram vector corresponding to each product image based on each mask contour, and normalize each chord angle histogram vector to obtain the local feature vector corresponding to each product image.

4. The method as described in claim 1, characterized in that, The step of performing image similarity matching based on the comprehensive feature vector within and between each of the image buckets, and obtaining a combination of images with the same style based on the matching results, includes: In each of the image buckets, similarity matching is performed based on the comprehensive feature vector corresponding to the product image to obtain the bucket matching result, and the optimal image combination within the bucket is determined based on the bucket matching result. The representative image corresponding to each image bucket is determined based on the optimal image combination within the bucket. Similarity matching is performed between the image buckets based on the comprehensive feature vectors corresponding to the representative images to obtain the bucket matching results; Based on the inter-bucket matching results, high-matching image buckets are determined, and similarity matching is performed based on the comprehensive feature vectors corresponding to the product images in the high-matching image buckets to obtain extended matching results; The optimal image combination is updated based on the extended matching results to obtain the same image combination.

5. The method as described in claim 4, characterized in that, The step of performing similarity matching based on the comprehensive feature vector corresponding to the product image in each of the image buckets to obtain the intra-bucket matching result, and determining the optimal image combination within the bucket based on the intra-bucket matching result, includes: Based on the image binning, the similarity between the comprehensive feature vectors corresponding to the product images is determined, and a similarity matrix is ​​obtained; Based on the similarity matrix, several similar image combinations are determined in descending order of similarity, and several similar image combinations that rank higher than a preset number are selected as candidate image combinations. Conflict detection is performed on each candidate image combination, and candidate image combinations with conflicts are eliminated to obtain the optimal image combination within the bucket.

6. The method as described in claim 4, characterized in that, The step of updating the optimal image combination based on the extended matching result to obtain the same image combination includes: The optimal image combination is updated based on the extended matching results to obtain the updated optimal image combination. The updated optimal image combination is then validated a second time according to the preset product parameter dimensions, and the same image combination is obtained based on the validation results.

7. The method as described in claim 1, characterized in that, The step of obtaining the current set of product images includes: Obtain the original set of product images, and sort the original set of product images according to the preset product category filtering rules to obtain the effective set of product images; The valid set of product images is subjected to image deduplication to obtain the current set of product images.

8. A product image matching device, characterized in that, The device includes: a memory, a processor, and a product image matching program stored in the memory and executable on the processor, the product image matching program being configured to implement the steps of the product image matching method as claimed in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium stores a matching program for product images, and when the matching program for product images is executed by the processor, it implements the steps of the matching method for product images as described in any one of claims 1 to 7.

Images