A method based on item similarity analysis

Abstract

This invention provides a method based on item similarity analysis, relating to the field of item similarity analysis technology, including the following steps: S1, item similarity calculation; S2, setting a mutual exclusion rule base; S3, dynamic GPU resource allocation. In this application, the operation process uses a deep convolutional network to extract the features of the items, judges the similarity between items based on the features, and uses the similarity between items and an experience base of easy identification errors as the item rule base to constrain the operation during replenishment, thereby avoiding identification errors caused by a large number of similar items in the cabinet. Then, the average similarity is calculated based on the similarity of the items in the cabinet, and the time for the algorithm to process the order video is predicted by combining the video length. The GPU resources are dynamically adjusted according to the overall time of the video to be processed.

Description

Technical Field

[0001] This invention relates to the field of item similarity analysis technology, and in particular to a method based on item similarity analysis. Background Technology

[0002] In the unmanned retail industry, item recognition relies on attributes such as the product's appearance, color, trademark information, and texture features. Due to the wide variety of items, many products share similar appearances, colors, and textures, making item recognition difficult. This is especially true in the realm of unmanned vending machines, where users may obstruct the view of items during handling, undoubtedly adding further challenges that current technologies cannot effectively address.

[0003] Meanwhile, product recognition in unmanned retail utilizes GPU resources for algorithmic processing. A given amount of GPU resources determines the algorithm's computing power, which in turn determines the number of videos that can be processed for recognition simultaneously. In practical applications, the number of videos to be recognized is not fixed, meaning the required GPU resources vary at different times. The inability to accurately calculate the required GPU resources can lead to either resource scarcity or waste. When GPU resources are scarce, it can cause processing delays for video orders, impacting user experience. Conversely, wasting GPU resources increases recognition costs. Summary of the Invention

[0004] The purpose of this invention is to provide a method based on item similarity analysis. It proposes constructing a rule base where similar products are mutually exclusive, prohibiting similar products from being placed in the same cabinet, thus constraining restocking. Simultaneously, it predicts the processing time for order videos based on existing historical data, thereby dynamically adjusting GPU resources.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method based on item similarity analysis, comprising the following steps:

[0006] S1. Product similarity calculation;

[0007] S2. Set up a mutual exclusion rule base;

[0008] S3, Dynamic GPU Resource Allocation.

[0009] Preferably, according to the operation steps in S1, the method for calculating product similarity is as follows:

[0010] S11. Similarity calculation is first performed using ResNet50 for training. The training data consists of 1000 types of real-world products, with 400 images for each type, and each image is 320×320 resolution.

[0011] S12. Image enhancement methods are used, and the images are cropped to a size of 256*256 along the center, upper left, lower left, upper right, and lower right. The cropped images are then horizontally flipped and rotated by 60 degrees and 30 degrees. Random noise, color perturbation, and random noise are added. Then, 1000 images of 1000 categories are randomly selected from the enhanced data to train the initial classification and recognition model.

[0012] Preferably, according to the operation steps in S2, the rule base refers to the rule that similar products are not allowed to be placed in the same cabinet. The rule base is divided into similarity exclusion and experience base.

[0013] Preferably, according to the operation steps in S3, the time taken by the algorithm to process order videos is closely related to the complexity of the goods and the length of the video, where the complexity of the goods can be defined as the overall recognition complexity (similarity) within the cabinet:

[0014] The overall complexity of identifying items within the cabinet is calculated by summing and averaging the similarities between the items in the cabinet. Assuming there are M items in the cabinet, the calculation is as follows:

[0015]

[0016] Where i≠j, n=(MD*M / 2;

[0017] Where v represents the feature vector, i ≠ j, i and j represent different feature vectors, n is the number of times the similarity between items in the unmanned vending machine is calculated, and the calculation formula is n = (M-1)*M / 2, f θ Here is the formula for calculating the dot product;

[0018] Generally, the average complexity within the cabinet is calculated after replenishment, and then written into the database for storage.

[0019] The algorithm's processing time is predicted based on the average complexity θ obtained from historical data. m Video duration t v And the actual algorithm processing time t p Given 3000 data entries, let the average complexity be θ. m Video duration t v Let X = {x1, x2} be the variable, and the actual processing time be t. p For Y = {t p The historical data is fitted nonlinearly to predict Y. The prediction uses a common nonlinear regression method, employing quadratic regression. The final prediction model is as follows:

[0020]

[0021] Tp (x1, x2) is the general formula for the quadratic of the prediction model, where x1 and x2 are the average complexity θ of the input, respectively. m Video duration t v Two variables, where a, b, c, d, f, and e are the coefficients of the quadratic term.

[0022] Nonlinear regression is usually transformed into linear regression and then solved according to linear regression. Linear regression usually uses the minimum sum of squares of the differences between the given function value and the model prediction value as the loss function, and uses the least squares method and gradient descent method to calculate the final fitting parameters. The model is solved using the machine learning sklearn library.

[0023] The final input is the video length t. r The complexity θ of the goods in the cabinet m To function T p (x1, x2) can be used to calculate the required time:

[0024] t p =T p (θ m , t v )

[0025] Let N be the number of orders to be identified in the current queue, and k be the number of GPU instances currently in use. The total prediction time for the current orders to be identified is:

[0026]

[0027] The number of orders to be identified is N, where i refers to a specific order among the current orders to be identified, and T is the total prediction time for calculating the current orders to be identified;

[0028] Based on experience, the maximum waiting time t for a single order is generally no more than 40 seconds. Calculate the number of GPU instances m required to identify the order:

[0029]

[0030] T is the sum of the prediction times for the current order to be identified, t is the maximum waiting time of 40 seconds, and m is the number of GPU instances required.

[0031] The formula for determining whether to add or remove GPU instances is as follows: Based on the number of GPU instances *m* required for the order to be identified and the actual number of online GPU instances *k*.

[0032]

[0033] Once the number of GPU instances that need to be added or reduced is calculated, the GPU service can be dynamically adjusted by calling the GPU elastic scaling interface.

[0034] Preferably, according to the operation steps in S12, the training parameters are:

[0035] The optimizer chosen is SGD, the loss function is cross-entropy loss, the initial learning rate is 0.1, the decay coefficient is 0.1, and the trained model is saved after training.

[0036] Modify the model by removing the softmax layer and adding PCA dimensionality reduction to reduce the 1000-dimensional feature vector to 512 dimensions;

[0037] The final feature vector is as follows:

[0038] v = f v (x)=f pca (f cnn (x))

[0039] Where x represents the input image pixels, f cnn This represents the 1*1000 dimensional feature vector v′ output by the ResNet50 convolutional layer, after passing through f... pca After dimensionality reduction, it becomes a 1*512 dimensional feature vector v;

[0040] The similarity calculation method uses the dot product of two feature vectors:

[0041] θ=f θ (v1, v2)

[0042] Where v1 and v2 are two feature vectors, f θ The dot product of two feature vectors yields an output θ, which represents the similarity between the two feature vectors.

[0043] Preferably, similarity and mutual exclusion are determined based on the product similarity calculation. When a new product is added to the product database, its features are extracted and its similarity is calculated with the products in the inventory. When the similarity is greater than 0.85, the product in the inventory is marked as a similar and mutually exclusive product.

[0044] The experience base is built by statistically analyzing after-sales records to identify products that are prone to misidentification. These products are then included in the experience rule base. The experience rule base mainly considers situations where, in real-world scenarios, two products have indistinct trademark features but the same color, or two originally dissimilar products have similar remaining features due to obscuring. Because these rules are difficult to define, they can only be obtained from past experience data on products that are prone to misidentification. The experience data mainly records the types and frequency of products that are prone to misidentification in after-sales. When the number of misidentifications exceeds a certain threshold N (set to 50 based on experience), the product is automatically added to the experience rule base.

[0045] This invention provides a method based on item similarity analysis. It has the following beneficial effects:

[0046] This method based on item similarity analysis uses a deep convolutional network to extract product features, determines the similarity between products based on these features, and uses the similarity between products and an experience base of easily misidentified products as a product rule base to constrain the replenishment operation. This can avoid identification errors caused by a large number of similar products in the cabinet. Then, the average similarity is calculated based on the similarity of products in the cabinet, and the time for the algorithm to process order videos is predicted by combining the video length. GPU resources are dynamically adjusted according to the total time of the videos to be processed. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0048] Figure 2 This is a diagram of the training model structure for similarity calculation in this invention;

[0049] Figure 3 This is a structural diagram of the similarity calculation and feature vector extraction model of the present invention;

[0050] Figure 4 This is a product mutual exclusion diagram of the present invention;

[0051] Figure 5 This is a diagram illustrating the mutual exclusion of Sprite, canned Sprite, plum green tea, and large bottle Sprite according to the present invention.

[0052] Figure 6 A flowchart illustrating the construction of similarity and mutual exclusion rules for this invention;

[0053] Figure 7 A flowchart for constructing the experience base of this invention. Detailed Implementation

[0054] This invention provides a technical solution: such as Figure 1-7 As shown, a method based on item similarity analysis includes the following specific steps:

[0055] Product similarity calculation:

[0056] Similarity calculation is first performed using ResNet50 for training, and the model structure is as follows: Figure 2 As shown;

[0057] The training data consisted of 1,000 types of real-world products, with 400 images for each type, and each image was 320×320 resolution.

[0058] Image enhancement methods were employed, and images were cropped to a size of 256*256 along the center, top left, bottom left, top right, and bottom right. The cropped images were then horizontally flipped and rotated by 60 degrees and 30 degrees. Random noise, color perturbation, and other random noise additions were also applied. These are common image data enhancement techniques and will not be elaborated upon. Subsequently, 1000 images from 1000 categories were randomly selected from the enhanced data to train an initial classification and recognition model. The training parameters were as follows:

[0059] The optimizer chosen is SGD, the loss function is cross-entropy loss, the initial learning rate is 0.1, the decay coefficient is 0.1, and the trained model is saved after training.

[0060] The model was then modified by removing the softmax layer and adding PCA dimensionality reduction to reduce the 1000-dimensional feature vector to 512 dimensions, as follows: Figure 3 As shown:

[0061] The final feature vector is as follows:

[0062] v = f v (x)=f pca (f cnn (x))

[0063] Where x represents the input image pixels, f cnm This represents the 1*1000 dimensional feature vector v′ output by the ResNet50 convolutional layer, after passing through f... pca After dimensionality reduction, it becomes a 1*512 dimensional feature vector v;

[0064] The similarity calculation method uses the dot product of two feature vectors:

[0065] θ=f θ (v1, v2)

[0066] Where v1 and v2 are two feature vectors, f θ The dot product of two feature vectors yields an output θ, which represents the similarity between the two feature vectors.

[0067] If the input images are two x1 and x2, then the similarity calculated by θ is:

[0068] θ=F θ (x1, x2) = f θ (f v (x1), f v (x2)).

[0069] Configure a mutual exclusion rule base:

[0070] The rule base refers to the rule that similar products are not allowed to be placed in the same cabinet. It can be defined as follows: A and B, C, and D are mutually exclusive. Figure 4 As shown,

[0071] For example, Sprite, canned Sprite, plum green tea, and large bottle Sprite are similar in appearance, color, trademark, and shape. Figure 5 As shown;

[0072] The rule base is divided into a similarity-exclusive rule base and an experience-based rule base. The similarity-exclusive rule base determines the similarity of products. The process for building similarity-exclusive rules is as follows: Figure 6 As shown:

[0073] When a new product is added to the product database, its features are extracted and its similarity is calculated with the products in the inventory. If the similarity is greater than 0.85, the product in the inventory is marked as a similar but mutually exclusive product.

[0074] The experience base is built upon after-sales records, statistically analyzing which products are prone to misidentification and incorporating them into the experience rule base. The experience rule base primarily considers real-world scenarios where two products have indistinct trademark features but share the same color, or where two originally dissimilar products have similar remaining features due to obscuring. Since rules for these situations are difficult to define, they can only be derived from past experience data on products prone to misidentification.

[0075] The experience data mainly records the types and frequency of products that are easily misidentified during after-sales service. When the number of misidentifications exceeds a certain threshold N (usually set to 50 based on experience), the data is automatically added to the experience rule base.

[0076] Experience base construction process: such as Figure 7 As shown;

[0077] The threshold N is generally set to 50 (derived from practical experience).

[0078] Dynamic GPU resource allocation

[0079] Algorithm processing

[0080] The time required to generate an order video is closely related to the complexity of the products and the length of the video. Product complexity can be defined as the overall recognition complexity (similarity) within the display case.

[0081] The overall complexity of identifying items within the cabinet is calculated by summing and averaging the similarities between the items in the cabinet. Assuming there are M items in the cabinet, the calculation is as follows:

[0082]

[0083] Where i ≠ j, n = (M-1)*M / 2

[0084] Where v represents the feature vector, i ≠ j, i and j represent different feature vectors, n is the number of times the similarity between items in the unmanned vending machine is calculated, and the calculation formula is n = (M-1)*M / 2, f θ This is the formula for calculating the dot product.

[0085] Typically, the average complexity within the cabinet is calculated after replenishment, and then written into the database for storage.

[0086] The algorithm's processing time is predicted based on the average complexity θ obtained from historical data. m Video duration t v And the actual algorithm processing time t p Given 3000 data entries, let the average complexity be θ. m Video duration t v Let X = {x1, x2} be the variable, and the actual processing time be t. p For Y = {t p The historical data is fitted nonlinearly to predict Y. A common nonlinear regression method is used, with quadratic regression employed. The final prediction model is as follows:

[0087]

[0088] T p (x1, x2) is the general formula for the quadratic of the prediction model, where x1 and x2 are the average complexity θ of the input, respectively. m Video duration t v Two variables, where a, b, c, d, f, and e are the coefficients of the quadratic term.

[0089] Nonlinear regression is typically transformed into linear regression and then solved using linear regression principles. Linear regression usually uses the minimum sum of squared differences between given function values ​​and model predictions as the loss function, and employs least squares and gradient descent to calculate the final fit parameters. The model is solved using the sklearn library for machine learning, as its operation is relatively simple and convenient, and will not be elaborated upon further.

[0090] The final input is the video length t. v The complexity θ of the goods in the cabinet m To function T p (x1, x2) can be used to calculate the required time:

[0091] t p =T p (θ m , t v )

[0092] Let N be the number of orders to be identified in the current queue, and k be the number of GPU instances currently in use. The total prediction time for the current orders to be identified is calculated as follows:

[0093]

[0094] The number of orders to be identified is N, where i refers to a specific order among the current orders to be identified, and T is the total prediction time for calculating the current orders to be identified;

[0095] Based on experience, the maximum waiting time t for a single order generally does not exceed 40 seconds. Calculate the number of GPU instances m required to identify the order:

[0096]

[0097] T is the sum of the prediction times for the current order to be identified, t is the maximum waiting time of 40 seconds, and m is the number of GPU instances required.

[0098] The formula for determining whether to add or remove GPU instances is as follows: Based on the number of GPU instances *m* required for the order to be identified and the actual number of online GPU instances *k*.

[0099]

[0100] Once the number of GPU instances that need to be added or reduced is calculated, the GPU service can be dynamically adjusted by calling the GPU elastic scaling interface.

[0101] The operation process involves using a deep convolutional network to extract product features, determining the similarity between products based on these features, and using the product similarity and a database of experiences that are prone to misidentification as a product rule base to constrain the replenishment operation. This helps avoid identification errors caused by a large number of similar products in the cabinet. Then, the average similarity is calculated based on the similarity of the products in the cabinet, and the time required for the algorithm to process order videos is predicted by combining this with the video length. GPU resources are dynamically adjusted based on the overall time of the videos to be processed.

[0102] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method of item similarity analysis based on, characterized by, Comprising the following steps: S1, commodity similarity calculation; S2, set mutual exclusion rule base; S3, dynamic GPU resource allocation; According to the operation steps in S3, the time consumption of the algorithm processing order video is closely related to the complexity of the commodity and the length of the video, wherein the complexity of the commodity can be defined as the overall recognition complexity in the cabinet: The overall recognition complexity in the cabinet is to calculate the similarity between the commodities in the cabinet and sum them up to get the average. Assuming that the number of commodities in the cabinet is M, the calculation is as follows: ； Where i≠j, n=(M-1)*M / 2; where v represents the eigenvector, i≠j, i and j represent different eigenvectors, n is the number of times of calculating the similarity between the goods in the unmanned container, the calculation formula is n=(M-1)*M / 2, f θ is the dot product calculation formula; The algorithm's processing time is predicted based on the average complexity θ obtained from historical data. m Video duration t v And the actual algorithm processing time t p Given 3000 data entries, let the average complexity be θ. m Video duration t v Let X = {x1, x2} be the variable, and the actual processing time be t. p For Y = {t p The historical data is fitted nonlinearly to predict Y. The prediction uses a common nonlinear regression method, specifically quadratic regression. The prediction model is as follows: ； T p (x1, x2) is a general formula of quadratic prediction model, where x1, x2 are the input average complexity θ m , the time length t v of the video, where a, b, c, d, f, e are the coefficients of the quadratic term; Nonlinear regression is usually converted to linear regression, and then solved according to linear regression. Linear regression usually uses the sum of the squares of the differences between the given function value and the model predicted value as the loss function, and uses the least squares method and gradient descent method to calculate the final fitting parameters. The model solution uses the machine learning sklearn library to solve; Input video length t v And the complexity of the goods in the cabinet θ m To the function T p (x1, x2), that is, the required time can be calculated: t p = T p (θ m , t v ) Let the number of orders to be identified in the current queue be N, the number of GPU instances currently in use be k, and the sum of the predicted time of the current order to be identified be: ； The number of orders to be identified is N, i refers to a certain order in the current order to be identified, and T is the sum of the predicted time of the current order to be identified; The maximum waiting time t of an order does not exceed 40s, and the number of GPU instances required by the order to be identified is m: M=T / t T is the sum of the predicted time of the current order to be identified, t is the maximum waiting time 40s, and m is the number of GPU instances required; According to the number of GPU instances required by the order to be identified m and the number of actual online GPU instances k, whether to add or reduce the number of GPU instances is calculated, and the formula is as follows: ； The number of GPU instances that needs to be added or reduced is calculated.

2. The method of claim 1, wherein, According to the operation steps in S1, the method for calculating the similarity of commodities is: S11, similarity calculation is trained using resnet50, and the training data is 1000 kinds of actual commodities, each kind has 400 pictures, and each picture is 320x320 resolution; S12, adopt image enhancement method, and cut along the center, upper left, lower left, upper right, and lower right with a size of 256*256, and perform horizontal flip, rotation by 60 degrees and 30 degrees, add random noise, color disturbance and random noise, then randomly select 1000 images from the enhanced data, and train the initial model for classification and identification.

3. The method of claim 1, wherein, According to the operation steps in S2, the rule base refers to the comparison of similar commodities, which is not allowed to be placed in the same cabinet. The rule base is divided into similar mutual exclusion and experience base.

4. The method of claim 2, wherein, According to the operation steps in S12, the training parameters are: Optimizer selects SGD, loss function adopts cross-entropy loss, initial learning rate is 0.1, and decreasing decay coefficient is 0.

1. After training, save the trained model; Modify the model to remove the softmax layer and add PCA dimension reduction to reduce the feature vector from 1000 dimensions to 512 dimensions; The final feature vector is as follows: v = f v (x) = f pca (f cnn (x)) where x represents the input image pixel, f cnn represents a 1*1000-dimensional feature vector v' output by the resnet50 convolutional layer, and f pca is reduced to a 1*512-dimensional feature vector v after dimension reduction processing. The similarity calculation method uses two feature vectors to multiply: θ = f θ (v1, v2) where v1, v2 are two eigenvectors, f θ To multiply two eigenvectors, the output result θ is the similarity of the two eigenvectors.

5. The method based on the analysis of the similarity of the goods according to claim 3, characterized in that, The similarity exclusion is determined according to the similarity calculation of the goods, when the goods in the warehouse are added, the feature extraction is performed on the goods, and the similarity calculation of the goods is performed on the goods in stock, when the similarity is greater than 0.85, the goods in stock are marked as similar exclusion goods; The experience library is statistically analyzed according to the record of the after-sales, which is easy to identify the error, and is included in the experience rule library, the experience rule library mainly considers that the trademark features of two goods are not obvious in the actual scene, but the colors are the same, or the features of the remaining part of the goods are similar because of the shielding of the originally dissimilar two goods, mainly records the types and times of the goods which are easy to be identified wrong in each goods in the after-sales, when the threshold N is set to 50, it is automatically added to the experience rule library.

Images