Commodity detection and classification method and device
By setting the detection layer and classification layer in the initial model, and optimizing the model parameters, the problem of low recognition efficiency in the existing technology is solved, and the SKU recognition of low labeling costs, high recognition accuracy and high efficiency SKU.
Patent Information
- Application Number
- CN202411739614.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-01-24
AI Technical Summary
The prior art is difficult to improve the efficiency of product SKU recognition while maintaining low labeling costs and high recognition accuracy, especially in the identification process of fast-moving consumer goods.
By setting the detection layer and classification layer in the initial model, and optimizing the model parameters based on gradient drop method, the product detection classification model is obtained. This model can detect and classify in a single model at the same time, avoid the complexity of the level joint model, and improve the recognition efficiency.
On the basis of low -marked costs and high recognition accuracy, the efficiency of product SKU recognition is improved, and it is suitable for fast -updated iterations such as fast -moving consumer goods.
Smart Images
Figure CN119357777A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of commodity detection and identification, and in particular to a commodity detection and classification method and device. Background Art
[0002] In the fast-moving consumer goods industry, the identification and inventory of product SKUs (Stock Keeping Units) is an important part of store warehouse goods management and supply chain visualization. Existing technologies generally use AI visual recognition technology to identify product SKUs. However, due to the large variety of SKUs of fast-moving consumer goods, small differences, and fast updates and iterations, the recognition model needs to ensure recognition speed while ensuring recognition accuracy, which is difficult.
[0003] There are two traditional product SKU identification solutions. The first is to build a single model to complete detection and classification at the same time. The second is to complete detection and classification by cascading the detection model and the classification (target detection) model. The former has high recognition accuracy and efficiency, but each time the product SKU is updated, additional annotations and retraining of the model are required. The annotation cost is high and the overall model training time is long. The latter only needs to update the annotation of the classification model every time the product SKU is updated. The annotation cost is low and the overall model training time is not high, but it cannot take into account both recognition accuracy and recognition efficiency. Therefore, how to improve recognition efficiency while maintaining low annotation costs and high recognition accuracy is still a problem that needs to be solved urgently in the existing technology. Summary of the invention
[0004] The present application provides a commodity detection and classification method and device to solve the technical problem that the prior art is difficult to achieve low labeling cost, high recognition accuracy and high recognition efficiency at the same time.
[0005] According to a first aspect of the embodiments of the present application, a commodity detection and classification method is provided, comprising:
[0006] Get the picture to be identified;
[0007] The image to be identified is input into a preset commodity detection and classification model to obtain and output the commodity detection and classification result; wherein the commodity detection and classification model is trained based on a preset initial model; the initial model sets a detection layer and a classification layer; the detection layer is used to detect and obtain multiple target areas; the classification layer is used to predict and output the commodity detection and classification result based on the image to be identified and the multiple target areas.
[0008] The present application first obtains the image to be identified, and then inputs the product detection and classification model to obtain and output the product detection and classification result. By setting the detection layer and the classification layer in the initial model and training to obtain the product detection and classification model, it is possible to maintain a low annotation cost, just like cascading the detection model and the classification model in the prior art. At the same time, compared with the cascading of the detection model and the classification model in the prior art, directly setting the detection layer and the classification layer in a single model without cascading the model can improve the recognition efficiency of the model, and has higher recognition accuracy than the single model that only performs detection in the prior art.
[0009] In certain embodiments of the present application, the step of inputting the image to be identified into a preset commodity detection classification model to obtain and output a commodity detection classification result specifically includes:
[0010] Extract features of the image to be identified and obtain a product feature map;
[0011] Based on a region recommendation network, performing region prediction on the product feature map to obtain multiple detection regions;
[0012] Performing feature pooling on the product feature map and the multiple detection areas to obtain multiple attention areas;
[0013] Performing target detection on the multiple attention areas through the detection layer to obtain multiple target areas;
[0014] Through the classification layer, the multiple target areas and the corresponding attention areas are combined to perform classification prediction to obtain a commodity detection classification result.
[0015] This application first extracts features from the image to be identified to obtain a product feature map, then performs region prediction based on a region recommendation network, and then performs feature pooling to obtain multiple attention regions, which can focus on each target product in the image to be identified, thereby improving the detection accuracy when performing target detection through the detection layer to obtain multiple target regions, thereby improving the classification accuracy when performing classification prediction through the classification layer combining multiple target regions and corresponding attention regions to obtain product detection classification results.
[0016] In certain embodiments of the present application, the commodity detection classification model is trained based on a preset initial model, specifically including:
[0017] Obtain detection branch dataset and classification branch dataset;
[0018] The detection branch data set and the classification branch data set are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the training is completed when the model parameters meet the preset convergence threshold, thereby obtaining a commodity detection classification model.
[0019] This application first obtains the detection branch data and the classification branch data set, and inputs the preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the model parameters meet the preset convergence threshold. The training is completed and the product detection and classification model is obtained. By continuously optimizing the model parameters through the gradient descent method, the model parameters can converge to the required range of the product detection and classification task, thereby obtaining a product detection and classification model that better meets the task requirements.
[0020] In certain embodiments of the present application, the acquiring of the detection branch dataset and the classification branch dataset specifically includes:
[0021] Obtain multiple detection images and multiple classification images;
[0022] Marking each target product in the multiple detection images, and obtaining a detection branch data set according to the multiple marked detection images;
[0023] Based on the preset side length, the multiple classification images are scaled in proportion to obtain multiple scaled images, and the multiple scaled images are placed into a preset background image to obtain a classification branch data set.
[0024] This application first obtains multiple detection images and multiple classification images, then marks each target product in the multiple detection images to obtain a detection branch data set, and at the same time, proportionally scales the multiple classification images based on a preset side length to obtain multiple scaled images, and places the multiple scaled images into a preset background image to obtain a classification branch data set. By preprocessing the images of the detection branch and the classification branch, the model can focus on the detection branch tasks and the classification branch tasks during model training, thereby obtaining a product detection and classification model that better meets the actual task requirements.
[0025] In certain embodiments of the present application, the detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training, and the model parameters of the initial model are optimized based on the gradient descent method during iteration, specifically including:
[0026] The detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss; wherein the overall training loss includes the region recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss; the detection branch loss is calculated based on the detection branch dataset; the classification branch loss is calculated based on the classification branch dataset; the feature metric loss is calculated based on the detection branch dataset and the classification branch dataset.
[0027] In this application, the detection branch dataset and the classification branch dataset are input into the initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss. The model is trained by the overall training loss including the regional recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss. This can improve the comprehensiveness of the model training evaluation, reasonably determine the model parameters of the commodity detection and classification model, and then obtain a commodity detection and classification model that better meets the actual task requirements, thereby improving the recognition accuracy when performing commodity detection and classification according to the commodity detection and classification model.
[0028] According to a second aspect of the implementation of the present application, there is provided a commodity detection and classification device, including a to-be-identified image acquisition module and a commodity detection and classification module;
[0029] The to-be-recognized picture acquisition module is used to acquire the to-be-recognized picture;
[0030] The commodity detection and classification module is used to input the image to be identified into a preset commodity detection and classification model to obtain and output the commodity detection and classification result; wherein the commodity detection and classification model is trained based on a preset initial model; the initial model sets a detection layer and a classification layer; the detection layer is used to detect and obtain multiple target areas; the classification layer is used to predict and output the commodity detection and classification result based on the image to be identified and the multiple target areas.
[0031] In certain embodiments of the present application, the commodity detection and classification module includes a feature extraction unit, a region prediction unit, a feature pooling unit, a target detection unit, and a classification prediction unit;
[0032] The feature extraction unit is used to extract features from the image to be identified to obtain a commodity feature map;
[0033] The region prediction unit is used to perform region prediction on the commodity feature map based on a region recommendation network to obtain multiple detection regions;
[0034] The feature pooling unit is used to perform feature pooling on the commodity feature map and the multiple detection areas to obtain multiple attention areas;
[0035] The target detection unit is used to perform target detection on the multiple attention areas through the detection layer to obtain multiple target areas;
[0036] The classification prediction unit is used to perform classification prediction through the classification layer in combination with the multiple target areas and the corresponding attention areas to obtain a commodity detection classification result.
[0037] In certain embodiments of the present application, the commodity detection classification model is trained based on a preset initial model, specifically including:
[0038] Obtain detection branch dataset and classification branch dataset;
[0039] The detection branch data set and the classification branch data set are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the training is completed when the model parameters meet the preset convergence threshold, thereby obtaining a commodity detection classification model.
[0040] In certain embodiments of the present application, the acquiring of the detection branch dataset and the classification branch dataset specifically includes:
[0041] Obtain multiple detection images and multiple classification images;
[0042] Marking each target product in the multiple detection images, and obtaining a detection branch data set according to the multiple marked detection images;
[0043] Based on the preset side length, the multiple classification images are scaled in proportion to obtain multiple scaled images, and the multiple scaled images are placed into a preset background image to obtain a classification branch data set.
[0044] In certain embodiments of the present application, the detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training, and the model parameters of the initial model are optimized based on the gradient descent method during iteration, specifically including:
[0045] The detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss; wherein the overall training loss includes the region recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss; the detection branch loss is calculated based on the detection branch dataset; the classification branch loss is calculated based on the classification branch dataset; the feature metric loss is calculated based on the detection branch dataset and the classification branch dataset.
[0046] The present application first obtains the image to be identified, and then inputs the product detection and classification model to obtain and output the product detection and classification result. By setting the detection layer and the classification layer in the initial model and training to obtain the product detection and classification model, it is possible to maintain a low annotation cost, just like cascading the detection model and the classification model in the prior art. At the same time, compared with the cascading of the detection model and the classification model in the prior art, directly setting the detection layer and the classification layer in a single model without cascading the model can improve the recognition efficiency of the model, and has higher recognition accuracy than the single model that only performs detection in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 : A flowchart of a commodity detection and classification method shown in certain embodiments of the present application;
[0048] Figure 2 : is a model structure diagram of a commodity detection classification model shown in certain embodiments of the present application;
[0049] Figure 3 : A module structure diagram of a commodity detection and classification device shown in certain embodiments of the present application. DETAILED DESCRIPTION
[0050] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application, and should not be construed as limitations on the embodiments of the present application. Based on the embodiments shown in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0051] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, unless otherwise clearly and specifically defined, "multiple" and "several" mean two or more.
[0052] In the traditional product SKU recognition solution, the recognition model that completes detection and classification work at the same time has high recognition accuracy and efficiency, but each time the product SKU is updated, it needs to be additionally labeled and retrained, which has high labeling costs and long overall model training time; the recognition model of the cascade detection model and classification model has low labeling costs and low overall model training time, but it is difficult to balance recognition accuracy and efficiency. Therefore, how to improve recognition efficiency while maintaining low labeling costs and high recognition accuracy is still a problem that needs to be solved urgently in the existing technology.
[0053] Based on the above technical background, please refer to Figure 1 The embodiment of the present application provides a commodity detection and classification method, including step S101 to step S102, each step is as follows:
[0054] Step S101: Obtain a picture to be identified.
[0055] Step S102: Input the image to be identified into a preset commodity detection and classification model to obtain and output the commodity detection and classification result; wherein the commodity detection and classification model is trained based on a preset initial model; the initial model sets a detection layer and a classification layer; the detection layer is used to detect and obtain multiple target areas; the classification layer is used to predict and output the commodity detection and classification result based on the image to be identified and the multiple target areas.
[0056] In certain embodiments of the present application, the step of inputting the image to be identified into a preset commodity detection classification model to obtain and output a commodity detection classification result specifically includes:
[0057] Extract features of the image to be identified and obtain a product feature map;
[0058] Based on a region recommendation network, performing region prediction on the product feature map to obtain multiple detection regions;
[0059] Performing feature pooling on the product feature map and the multiple detection areas to obtain multiple attention areas;
[0060] Performing target detection on the multiple attention areas through the detection layer to obtain multiple target areas;
[0061] Through the classification layer, the multiple target areas and the corresponding attention areas are combined to perform classification prediction to obtain a commodity detection classification result.
[0062] This application first extracts features from the image to be identified to obtain a product feature map, then performs region prediction based on a region recommendation network, and then performs feature pooling to obtain multiple attention regions, which can focus on each target product in the image to be identified, thereby improving the detection accuracy when performing target detection through the detection layer to obtain multiple target regions, thereby improving the classification accuracy when performing classification prediction through the classification layer combining multiple target regions and corresponding attention regions to obtain product detection classification results.
[0063] In certain embodiments of the present application, the commodity detection classification model is trained based on a preset initial model, specifically including:
[0064] Obtain detection branch dataset and classification branch dataset;
[0065] The detection branch data set and the classification branch data set are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the training is completed when the model parameters meet the preset convergence threshold, thereby obtaining a commodity detection classification model.
[0066] See also Figure 2 , which is a model structure diagram of a commodity detection and classification model shown in certain embodiments of the present application; wherein the input is a detection branch data set and a classification branch data set, Backbone is a feature extraction layer, RPN (Region Proposal Networks) is a region recommendation network, ROI Pooling (Region of Interest Pooling) is a feature pooling layer, Detect is a detection layer, and Classifier is a classification layer.
[0067] In certain embodiments of the present application, a preferred implementation of the feature extraction layer Backbone includes but is not limited to Swin Transformer; a preferred implementation of the detection layer Detect is two fully connected layers connected in parallel, wherein one fully connected layer is used for target area position regression and the other fully connected layer is used for target area determination; a preferred implementation of the classification layer Classifier is a fully connected layer for combining the outputs of the detection layer Detect and the feature pooling layer ROIPooling for classification prediction.
[0068] This application first obtains the detection branch data and the classification branch data set, and inputs the preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the model parameters meet the preset convergence threshold. The training is completed and the product detection and classification model is obtained. By continuously optimizing the model parameters through the gradient descent method, the model parameters can converge to the required range of the product detection and classification task, thereby obtaining a product detection and classification model that better meets the task requirements.
[0069] In certain embodiments of the present application, the acquiring of the detection branch dataset and the classification branch dataset specifically includes:
[0070] Obtain multiple detection images and multiple classification images;
[0071] Marking each target product in the multiple detection images, and obtaining a detection branch data set according to the multiple marked detection images;
[0072] Based on the preset side length, the multiple classification images are scaled in proportion to obtain multiple scaled images, and the multiple scaled images are placed into a preset background image to obtain a classification branch data set.
[0073] In certain embodiments of the present application, the pixel value of the preset side length is a whole ten, and the interval range is [40,300]; the size of the preset background image is 1333*800; and the multiple scaled images are spaced 8 pixels apart when placed in the preset background image.
[0074] In certain embodiments of the present application, when obtaining a detection branch data set based on a plurality of marked detection images, the position and category name of the marking box of each target product is used as the target annotation true value; when placing the plurality of zoomed images into a preset background image to obtain a classification branch data set, the position and category name of the zoomed image corresponding to each target product is used as the target annotation true value.
[0075] This application first obtains multiple detection images and multiple classification images, then marks each target product in the multiple detection images to obtain a detection branch data set, and at the same time, proportionally scales the multiple classification images based on a preset side length to obtain multiple scaled images, and places the multiple scaled images into a preset background image to obtain a classification branch data set. By preprocessing the images of the detection branch and the classification branch, the model can focus on the detection branch tasks and the classification branch tasks during model training, thereby obtaining a product detection and classification model that better meets the actual task requirements.
[0076] In certain embodiments of the present application, the detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training, and the model parameters of the initial model are optimized based on the gradient descent method during iteration, specifically including:
[0077] The detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss; wherein the overall training loss includes the region recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss; the detection branch loss is calculated based on the detection branch dataset; the classification branch loss is calculated based on the classification branch dataset; the feature metric loss is calculated based on the detection branch dataset and the classification branch dataset.
[0078] In certain embodiments of the present application, a preferred implementation of the overall training loss is the sum of the region proposal network loss, the detection branch loss, the classification branch loss and the feature metric loss.
[0079] In certain embodiments of the present application, the region recommendation network loss includes a region recommendation network regression loss and a region recommendation network classification loss; the detection branch loss includes a detection branch box regression loss and a detection branch box classification loss; the region recommendation network regression loss and the detection branch box regression loss are L1 losses; the region recommendation network classification loss, the detection branch box classification loss and the classification branch loss are cross entropy losses; the feature metric loss is calculated based on cosine similarity, specifically:
[0080]
[0081] Among them, metric_loss is the feature metric loss, fea1 and fea2 are the first ROI feature and the second ROI feature respectively.
[0082] Specifically, the first ROI feature and the second ROI feature are:
[0083] Through the feature extraction layer Backbone, feature extraction is performed on the detection branch data set to obtain a first detection feature map;
[0084] Through the feature pooling layer ROI Pooling, feature pooling is performed on the first detection feature map and the corresponding target annotation true value to obtain a first ROI feature;
[0085] The area in the detection branch data set that does not belong to the target annotation true value is set to black, and feature extraction is performed to obtain a second detection feature map;
[0086] Feature pooling is performed on the second detection feature map and the corresponding target annotation true value to obtain a second ROI feature.
[0087] In this application, the detection branch dataset and the classification branch dataset are input into the initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss. The model is trained by the overall training loss including the regional recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss. This can improve the comprehensiveness of the model training evaluation, reasonably determine the model parameters of the commodity detection and classification model, and then obtain a commodity detection and classification model that better meets the actual task requirements, thereby improving the recognition accuracy when performing commodity detection and classification according to the commodity detection and classification model.
[0088] Compared with the prior art, the present application first obtains the image to be identified, then inputs the product detection and classification model, obtains and outputs the product detection and classification result, and obtains the product detection and classification model by setting the detection layer and the classification layer in the initial model and training it. It can maintain a low annotation cost just like cascading the detection model and the classification model in the prior art. At the same time, compared with the cascading of the detection model and the classification model in the prior art, directly setting the detection layer and the classification layer in a single model without cascading the model can improve the recognition efficiency of the model, and has higher recognition accuracy than the single model that only performs detection in the prior art.
[0089] Corresponding to the previous method, see Figure 3 , the embodiment of the present application provides a commodity detection and classification device, including a to-be-identified image acquisition module 310 and a commodity detection and classification module 320;
[0090] The to-be-recognized picture acquisition module 310 is used to acquire the to-be-recognized picture;
[0091] The commodity detection and classification module 320 is used to input the image to be identified into a preset commodity detection and classification model to obtain and output the commodity detection and classification result; wherein the commodity detection and classification model is trained based on a preset initial model; the initial model sets a detection layer and a classification layer; the detection layer is used to detect and obtain multiple target areas; the classification layer is used to predict and output the commodity detection and classification result based on the image to be identified and the multiple target areas.
[0092] In certain embodiments of the present application, the commodity detection and classification module 320 includes a feature extraction unit, a region prediction unit, a feature pooling unit, a target detection unit, and a classification prediction unit;
[0093] The feature extraction unit is used to extract features from the image to be identified to obtain a commodity feature map;
[0094] The region prediction unit is used to perform region prediction on the commodity feature map based on a region recommendation network to obtain multiple detection regions;
[0095] The feature pooling unit is used to perform feature pooling on the commodity feature map and the multiple detection areas to obtain multiple attention areas;
[0096] The target detection unit is used to perform target detection on the multiple attention areas through the detection layer to obtain multiple target areas;
[0097] The classification prediction unit is used to perform classification prediction through the classification layer in combination with the multiple target areas and the corresponding attention areas to obtain a commodity detection classification result.
[0098] In certain embodiments of the present application, the commodity detection classification model is trained based on a preset initial model, specifically including:
[0099] Obtain detection branch dataset and classification branch dataset;
[0100] The detection branch data set and the classification branch data set are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the training is completed when the model parameters meet the preset convergence threshold, thereby obtaining a commodity detection classification model.
[0101] In certain embodiments of the present application, the acquiring of the detection branch dataset and the classification branch dataset specifically includes:
[0102] Obtain multiple detection images and multiple classification images;
[0103] Marking each target product in the multiple detection images, and obtaining a detection branch data set according to the multiple marked detection images;
[0104] Based on the preset side length, the multiple classification images are scaled in proportion to obtain multiple scaled images, and the multiple scaled images are placed into a preset background image to obtain a classification branch data set.
[0105] In certain embodiments of the present application, the detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training, and the model parameters of the initial model are optimized based on the gradient descent method during iteration, specifically including:
[0106] The detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss; wherein the overall training loss includes the region recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss; the detection branch loss is calculated based on the detection branch dataset; the classification branch loss is calculated based on the classification branch dataset; the feature metric loss is calculated based on the detection branch dataset and the classification branch dataset.
[0107] The present application first obtains the image to be identified, and then inputs the product detection and classification model to obtain and output the product detection and classification result. By setting the detection layer and the classification layer in the initial model and training to obtain the product detection and classification model, it is possible to maintain a low annotation cost, just like cascading the detection model and the classification model in the prior art. At the same time, compared with the cascading of the detection model and the classification model in the prior art, directly setting the detection layer and the classification layer in a single model without cascading the model can improve the recognition efficiency of the model, and has higher recognition accuracy than the single model that only performs detection in the prior art.
[0108] It should be understood that the device provided in the embodiments of the present application corresponds to the aforementioned method, and a commodity detection and classification device provided in the embodiments of the present application can implement a commodity detection and classification method provided in any embodiment of the present application.
[0109] Adaptively, the embodiments of the present application also provide a computer device and a computer-readable storage medium.
[0110] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;
[0111] Wherein, when the processor executes the computer program, a commodity detection and classification method of the present application is implemented.
[0112] The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute a commodity detection and classification method of the present application.
[0113] The above are some implementation methods of the present application, and the purpose, technical scheme and beneficial effects of the present application are further described in detail. It should be clear that the above-mentioned partial implementation methods of the present application cannot be understood as limiting the present application. It is particularly pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions and variations made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A commodity detection and classification method, characterized in that: include: Get the picture to be identified; Input the image to be identified into a preset commodity detection classification model to obtain and output a commodity detection classification result; wherein the commodity detection classification model is trained based on a preset initial model; The initial model sets a detection layer and a classification layer; the detection layer is used to detect and obtain multiple target areas; the classification layer is used to predict and output the commodity detection classification result based on the image to be identified and the multiple target areas.
2. A commodity detection and classification method according to claim 1, characterized in that: The step of inputting the image to be identified into a preset commodity detection and classification model to obtain and output the commodity detection and classification result specifically includes: Extract features of the image to be identified and obtain a product feature map; Based on a region recommendation network, performing region prediction on the product feature map to obtain multiple detection regions; Performing feature pooling on the product feature map and the multiple detection areas to obtain multiple attention areas; Performing target detection on the multiple attention areas through the detection layer to obtain multiple target areas; Through the classification layer, the multiple target areas and the corresponding attention areas are combined to perform classification prediction to obtain a commodity detection classification result.
3. A commodity detection and classification method according to claim 1, characterized in that: The commodity detection classification model is trained based on a preset initial model, and specifically includes: Obtain detection branch dataset and classification branch dataset; The detection branch data set and the classification branch data set are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the training is completed when the model parameters meet the preset convergence threshold, thereby obtaining a commodity detection classification model.
4. A commodity detection and classification method according to claim 3, characterized in that: The obtaining of the detection branch data set and the classification branch data set specifically includes: Obtain multiple detection images and multiple classification images; Marking each target product in the multiple detection images, and obtaining a detection branch data set according to the multiple marked detection images; Based on the preset side length, the multiple classification images are scaled in proportion to obtain multiple scaled images, and the multiple scaled images are placed into a preset background image to obtain a classification branch data set.
5. A commodity detection and classification method according to claim 3, characterized in that: The step of inputting the detection branch data set and the classification branch data set into a preset initial model for iterative training, and optimizing the model parameters of the initial model based on a gradient descent method during iteration, specifically includes: The detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss; wherein the overall training loss includes the region recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss; the detection branch loss is calculated based on the detection branch dataset; the classification branch loss is calculated based on the classification branch dataset; the feature metric loss is calculated based on the detection branch dataset and the classification branch dataset.
6. A commodity detection and classification device, characterized in that: It includes a module for obtaining images to be identified and a module for detecting and classifying commodities; The to-be-recognized picture acquisition module is used to acquire the to-be-recognized picture; The commodity detection and classification module is used to input the image to be identified into a preset commodity detection and classification model to obtain and output the commodity detection and classification result; wherein the commodity detection and classification model is trained based on a preset initial model; the initial model sets a detection layer and a classification layer; the detection layer is used to detect and obtain multiple target areas; the classification layer is used to predict and output the commodity detection and classification result based on the image to be identified and the multiple target areas.
7. A commodity detection and classification device according to claim 6, characterized in that: The commodity detection and classification module includes a feature extraction unit, a region prediction unit, a feature pooling unit, a target detection unit and a classification prediction unit; The feature extraction unit is used to extract features from the image to be identified to obtain a commodity feature map; The region prediction unit is used to perform region prediction on the commodity feature map based on a region recommendation network to obtain multiple detection regions; The feature pooling unit is used to perform feature pooling on the commodity feature map and the multiple detection areas to obtain multiple attention areas; The target detection unit is used to perform target detection on the multiple attention areas through the detection layer to obtain multiple target areas; The classification prediction unit is used to perform classification prediction through the classification layer in combination with the multiple target areas and the corresponding attention areas to obtain a commodity detection classification result.
8. A commodity detection and classification device according to claim 6, characterized in that: The commodity detection classification model is trained based on a preset initial model, and specifically includes: Obtain detection branch dataset and classification branch dataset; The detection branch data set and the classification branch data set are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method until the training is completed when the model parameters meet the preset convergence threshold, thereby obtaining a commodity detection classification model.
9. A commodity detection and classification device according to claim 8, characterized in that: The obtaining of the detection branch data set and the classification branch data set specifically includes: Obtain multiple detection images and multiple classification images; Marking each target product in the multiple detection images, and obtaining a detection branch data set according to the multiple marked detection images; Based on the preset side length, the multiple classification images are scaled in proportion to obtain multiple scaled images, and the multiple scaled images are placed into a preset background image to obtain a classification branch data set.
10. A commodity detection and classification device according to claim 8, characterized in that: The step of inputting the detection branch data set and the classification branch data set into a preset initial model for iterative training, and optimizing the model parameters of the initial model based on a gradient descent method during iteration, specifically includes: The detection branch dataset and the classification branch dataset are input into a preset initial model for iterative training. During the iteration, the model parameters of the initial model are optimized based on the gradient descent method according to the calculated overall training loss; wherein the overall training loss includes the region recommendation network loss, the detection branch loss, the classification branch loss and the feature metric loss; the detection branch loss is calculated based on the detection branch dataset; the classification branch loss is calculated based on the classification branch dataset; the feature metric loss is calculated based on the detection branch dataset and the classification branch dataset.