Product Identification System

Abstract

A product identification system, the system comprising: a photographing device including a placement portion and a camera, the placement portion being used to place a product to be identified and having multiple placement areas, the camera being used to photograph the placement portion to obtain a first image; a client being used to preprocess the first image and receive an identification result, wherein the preprocessing includes: determining the product category of the product to be identified based on the placement area where the product to be identified is located, recorded as a target category; a server being used to select a target feature extraction model associated with the target category from multiple first feature extraction models, input at least a portion of the first image into the target feature extraction model to obtain first feature information, determine an identification result based on the first feature information, and send the identification result to the client. The solution provided by the present invention can improve the accuracy of product identification.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a product identification system. Background Art

[0002] As a new round of global scientific and technological revolution and industrial transformation rapidly advances, breakthroughs in next-generation information and communications, biotechnology, new materials, and new energy technologies are continuously emerging, and their integration with advanced manufacturing technologies is accelerating. This presents a historic opportunity for high-end, intelligent, and green development in the manufacturing industry. Automation and intelligentization of production processes are becoming increasingly prevalent, and AI-related algorithms are increasingly being implemented to empower industrial reform.

[0003] Currently, product quality inspection and classification are still primarily done manually. As product categories become increasingly refined, manual identification can easily lead to product confusion and misjudgment. Therefore, the accuracy of product identification needs to be improved. Summary of the Invention

[0004] The technical problem solved by the present invention is to provide a product identification system to improve the accuracy of product identification.

[0005] To solve the above technical problems, an embodiment of the present invention provides a product identification system, which includes: a shooting device, the shooting device includes a placement part and a camera, the placement part is used to place the product to be identified and has multiple placement areas, and the camera is used to shoot the placement part to obtain a first image; a client, used to preprocess the first image and receive the recognition result, wherein the preprocessing includes: determining the product category of the product to be identified according to the placement area where the product to be identified is located, recorded as the target category, and the placement area corresponds one-to-one to the product category; a server, used to select a target feature extraction model that has an association relationship with the target category from multiple feature extraction models, and input at least a part of the first image into the target feature extraction model to obtain first feature information, and determine the recognition result according to the first feature information, and send the recognition result to the client.

[0006] Optionally, each placement area has an indicator line and a visual label, wherein the indicator line is used to determine the range of the placement area, and the visual label is used to indicate the product category corresponding to the placement area.

[0007] Optionally, the photographing device further includes: a placement box, wherein the placement portion and the camera are located inside the placement box; wherein at least a portion of the placement box is opaque.

[0008] Optionally, at least a portion of the placement box is white.

[0009] Optionally, the photographing device further includes: a lighting device, wherein the lighting device is located inside the placement box.

[0010] Optionally, the target category belongs to a first category group or a second category group, wherein the first category group includes at least one first product category, and the number of sample images of each first product category is greater than a first preset threshold, and the second category group includes multiple second product categories, and the number of sample images corresponding to each second product category is less than or equal to the first preset threshold; the multiple feature extraction models include: a general feature extraction model and at least one exclusive feature extraction model, wherein the general feature extraction model is trained using sample images corresponding to the multiple second product categories as training data, and the exclusive feature extraction model corresponds one-to-one to the first product category, and each exclusive feature extraction model is trained using sample images corresponding to the corresponding first product category as training data.

[0011] Optionally, the at least one first product category includes one or more of the following: rings, necklaces, and bracelets; and the multiple second product categories include: brooches, earrings, and headwear.

[0012] Optionally, preprocessing the first image includes: if the target category is necklace, performing image detection on the first image to obtain a detection result, and the detection result is used to indicate whether the product to be identified is placed according to preset requirements; if the detection result indicates that the product to be identified is not placed according to preset requirements, reminding the user to adjust the placement status of the product to be identified.

[0013] Optionally, the recognition result is the SKU to which the product to be identified belongs, and the client is further used to obtain a second sample image and send the second sample image to the server; the server is further used to use at least a part of the first sample image and the second sample image as training data to train the feature extraction model to update the feature extraction model; wherein, the first sample image and the second sample image both have pre-labeled labels, and the labels are used to indicate the SKU to which the product in the sample image belongs, and the first sample image is a sample image used to train the feature extraction model before the update.

[0014] Optionally, before using at least a portion of the first sample image and the second sample image as training data to train the feature extraction model, the server side is also used to: determine whether a preset incremental training condition is met; if so, use the second sample image and at least a portion of the first sample image as training data to train the feature extraction model; wherein the incremental training condition includes one or more of the following: the system time reaches the preset training time, and the number of the second sample images reaches a second preset threshold.

[0015] Optionally, the identification result is the SKU to which the product to be identified belongs, and the system further includes: a database module, the database module including: multiple storage units, the storage units corresponding one-to-one to the product categories, and each storage unit is used to store characteristic information of the SKU belonging to the corresponding product category.

[0016] Optionally, the feature information of each SKU is calculated based on a feature vector associated with the SKU, wherein the feature vector associated with the SKU is obtained by performing feature extraction on a sample image containing products belonging to the SKU using the feature extraction model.

[0017] Optionally, before inputting at least a portion of the first image into the target feature extraction model, the server is further configured to perform standardization on the received image according to a preset size, wherein the preset size is determined based on the target category.

[0018] Optionally, the server side includes: a calling module, wherein the calling module is used to provide an interface to the client side to implement data interaction between the client side and the server side.

[0019] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0020] In the solution of the embodiment of the present invention, the shooting device includes a placement unit and a camera, wherein the placement unit has multiple placement areas, and the camera shoots the placement unit to obtain a first image; the client determines the placement area where the product to be identified is located based on the first image, and then determines the target category of the product to be identified based on the placement area where the product to be identified is located. Furthermore, the server determines a target feature extraction model based on the target category of the product to be identified, and inputs the received image into the target feature extraction model to obtain first feature information, and then determines the recognition result based on the first feature information. Compared with the existing solution using manual recognition, the solution provided by the embodiment of the present invention can more accurately determine the category of the product to be identified.

[0021] Furthermore, in an embodiment of the present invention, each placement area can have an indicator line and a visual label. The indicator line can be used to define the placement area, and the visual label can be used to indicate the product category corresponding to the placement area. This solution can guide users to place the product to be identified in the correct and appropriate location, thereby improving the accuracy of subsequent identification.

[0022] Furthermore, in the embodiment of the present invention, a unique feature extraction model is trained for each product category with a large number of sample images, while a universal feature extraction model is trained for product categories with a smaller number of sample images. This approach maximizes product recognition accuracy even when the number of sample images corresponding to different product categories is uneven.

[0023] Furthermore, in the solution of the embodiment of the present invention, when performing incremental training, the second sample image and a portion of the first sample image are used as training data, which is beneficial for the updated feature extraction model to have better expressive ability on both new and old training data, thereby helping to improve the accuracy of product recognition.

[0024] Furthermore, in the solution of the embodiment of the present invention, the identification results of the product to be identified are first determined in the historical SKU. If a certain number of identification results or a high degree of accuracy cannot be determined in the historical SKU, the identification results of the product to be identified are then determined in combination with the newly added SKU. This solution can minimize conflicts under high concurrency and improve recognition efficiency while ensuring accuracy as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the hardware structure of a product identification system according to an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of the three-dimensional structure of a shooting device according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the relationship between modules in a product identification system according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the architecture of a product identification system according to an embodiment of the present invention;

[0029] Figure 5 It is a flowchart of a product identification method in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] As described in the background art, the accuracy of product identification in the prior art needs to be improved.

[0031] In order to solve the above technical problems, an embodiment of the present invention provides a product identification system. In the solution of the embodiment of the present invention, the shooting device includes a placement part and a camera, wherein the placement part has multiple placement areas, and the camera shoots the placement part to obtain a first image; the client determines the placement area where the product to be identified is located based on the first image, and then determines the target category of the product to be identified based on the placement area where the product to be identified is located. Furthermore, the server determines a target feature extraction model based on the target category of the product to be identified, and inputs the received image into the target feature extraction model to obtain first feature information, and then determines the identification result based on the first feature information. Compared with the existing solution of manual identification, the solution provided by the embodiment of the present invention can more accurately determine the category of the product to be identified.

[0032] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0033] Reference Figure 1 , Figure 1 This is a schematic diagram of the hardware structure of a product identification system according to an embodiment of the present invention. The product identification system according to the embodiment of the present invention can be used to identify the smallest stock keeping unit (SKU) to which a product belongs. In other words, when the product identification system according to the embodiment of the present invention is used to identify a product to be identified, the identification result obtained is the SKU to which the product to be identified belongs.

[0034] The following combination Figure 1 The product identification system in the embodiment of the present invention is described in a non-limiting manner.

[0035] like Figure 1 As shown, the product identification system 1 in the embodiment of the present invention may include: a server 11, a client 12, and a camera 13. The server 11 may include one or more servers, and the server 11 is coupled to at least one client 12. Each client 12 may be coupled to at least one camera 13. "Coupling" in the embodiment of the present invention may refer to a communication connection. The client 12 may refer to a terminal used by a user, which may be any terminal with data receiving and data processing capabilities, such as, but not limited to, a computer, a mobile phone, a tablet computer, and an IoT device.

[0036] It should be noted that the embodiment of the present invention does not limit the number of clients 12 coupled to the server 11 , nor does it limit the number of photographing devices coupled to each client 12 .

[0037] In the embodiment of the present invention, the camera 13 can be used to photograph products. Specifically, during use of the product identification system 1, the camera 13 can be used to photograph products to be identified; during the establishment of the product identification system, the camera 13 can be used to photograph registered products. The products to be identified may refer to products whose SKUs are unknown, while registered products may refer to products whose SKUs are known.

[0038] Reference Figure 2 , Figure 2 This is a schematic diagram of the three-dimensional structure of a shooting device according to an embodiment of the present invention. Figure 2 The photographing device is described in a non-limiting manner.

[0039] Specifically, the photographing device 13 may include a placement box 130, a placement portion 131, and a camera 132, wherein the placement portion 131 and the camera 132 may be located inside the placement box 131. In the embodiment of the present invention, the type of the camera 132 is not limited, and may be any existing camera.

[0040] The placement portion 131 can be used to place a product to be identified (not shown), and the camera 132 can be used to capture the placement portion 131 to obtain a first image, wherein the first image can include an image of the product to be identified. Accordingly, when a registered product is placed on the placement portion 131, the camera 132 captures the placement portion 131 to obtain a sample image, wherein the sample image includes an image of the registered product.

[0041] More specifically, the placement portion 131 has a plurality of placement areas (not shown), and the placement areas may correspond one to one with the product categories of the products. During use of the product identification system, the user may place the product to be identified in the placement area corresponding to the product category of the product to be identified.

[0042] It should be noted that in the embodiments of the present invention, product category refers to the type of product. For example, a product identification system can be used to identify accessories, and product categories may include, but are not limited to, rings, necklaces, bracelets, earrings, and headwear. For another example, a product identification system can be used to identify clothing, and product categories may include, but are not limited to, tops, dresses, bottoms, and shoes. Furthermore, each product category includes multiple SKUs, and different product categories include different SKUs.

[0043] In the solution of the embodiment of the present invention, the multiple placement areas may not overlap with each other.

[0044] In a specific implementation, each placement area can have an indicator line and a visual label. The indicator line can be used to define the placement area, and the visual label can be used to indicate the product category corresponding to the placement area. This solution can guide users to place the product to be identified in the correct and appropriate location, thereby improving the accuracy of subsequent identification.

[0045] In one non-limiting embodiment, the placement areas further comprise mold bases, and the placement areas and the mold bases are in a corresponding relationship, with different placement areas having different mold bases. More specifically, the shape of the mold base corresponding to each placement area is adapted to the shape of the product of the product category corresponding to that placement area. During the image capture, the product to be identified is placed on the mold base.

[0046] like Figure 2 As shown, the placement box 130 can be a box body, and when the product to be identified is photographed, the placement box 130 is in a closed state. Further, in order to improve the quality of the image obtained by photographing, at least a portion of the placement box 130 can be opaque.

[0047] In a specific example, the placement box 130 may include a light-shielding portion (not shown) and a light-transmitting portion (not shown). The area of ​​the light-shielding portion may be larger than the area of ​​the light-transmitting portion, wherein the light-transmitting portion may be transparent to facilitate the user to observe the placement status of the product to be identified; the light-shielding portion may be opaque, which may help avoid the influence of light on the quality of the captured image.

[0048] In a non-limiting example, the placement box 130 may be in the shape of a cuboid, wherein five faces of the cuboid may be opaque, one face may be transparent, or all six faces may be opaque.

[0049] Furthermore, the light shielding portion of the placement box 130 can be white. By adopting such a solution, the influence of the color of the inner surface of the placement box 130 on the image can be reduced as much as possible, which is conducive to improving the quality of the image, thereby improving the accuracy of recognition.

[0050] Furthermore, the camera 13 may also include a lighting device 133, which is also located inside the storage box 130. The lighting device 133 may be, but is not limited to, an LED light. More specifically, the brightness of the lighting device 133 is adjustable. For example, the camera 13 may include a knob, which the user can use to adjust the brightness of the lighting device 133.

[0051] It should be noted that the relative positions of the lighting device 133, the camera 132, and the placement portion 131 can be determined based on actual conditions and are not limited in this embodiment. It should also be noted that the position and shooting angle of the camera 132 must be able to capture all placement areas so that the client can subsequently determine the placement area where the product to be identified is located.

[0052] In a non-limiting example, the placement portion 131 can be located at the bottom of the placement box 130, and the camera 132 can be located at the top of the placement box 130. More specifically, the camera 132 can be centrally located at the top of the placement box 130, and the lighting devices 133 are symmetrically provided on both sides of the camera 132.

[0053] Continue to refer Figure 1 During use of the product identification system 1 , after the user places the product to be identified on the placement portion, the camera can capture the placement portion to obtain a first image, and send the first image to the client 12 .

[0054] Furthermore, the client 12 may pre-process the first image.

[0055] Specifically, the pre-processing performed by the client 12 on the first image may include: determining the product category of the product to be identified based on the placement area of ​​the product to be identified. For ease of description, the product category of the product to be identified is referred to as the target category below.

[0056] More specifically, since the user places the product to be identified in the corresponding placement area based on its category and uses a camera to capture the placement area, the client 12 can determine the position of the image of the product to be identified in the first image after receiving the first image, and thus determine the placement area where the product to be identified is located. An object detection algorithm can be used to identify the position of the image of the product to be identified in the first image. The object detection algorithm can be any suitable existing object detection algorithm, such as an RGB algorithm (e.g., an edge detection algorithm and a contour detection algorithm), an image binarization algorithm, etc.

[0057] As described above, the product recognition system 1 can be used to identify the SKU to which an item of jewelry belongs. In this application scenario, if the target category is determined to be a necklace, image detection can be performed on the first image to obtain a detection result. This detection result can be used to indicate whether the item to be identified is placed according to preset requirements. If so, subsequent steps can be performed. If the detection result indicates that the item to be identified is not placed according to preset requirements, a first reminder message can be issued, which can be used to remind the user to adjust the placement of the item to be identified. The first reminder message can be a voice message, a graphic message, or other information. This embodiment does not limit the type of the first reminder message. Since necklace products typically include a pendant and a chain, the same pendant but a different chain will result in a different SKU for the necklace product. Therefore, the chain also has a significant impact on the recognition result. If the chain is placed arbitrarily, it can easily lead to large errors in the recognition of the chain, resulting in an erroneous final recognition result. Therefore, the above solution can ensure the correct placement of necklace products, thereby improving the accuracy of product recognition.

[0058] In a specific example, before determining the target category, the client 11 can first use an image quality diagnosis algorithm to determine whether the image quality is qualified. If so, preprocessing can continue. Otherwise, the client 11 can send a second reminder message to the user, which can be used to remind the user to check the camera 13 and reposition the product to be identified after inspection. The image quality diagnosis algorithm can be a blind / referenceless image spatial quality evaluator (BRISQUE) algorithm, but is not limited to this.

[0059] Furthermore, the client 12 may send the area of ​​the first image containing the image of the product to be identified to the server 11, and also send the target category to the server 11. For ease of description, the image sent by the client 12 is hereinafter referred to as the second image.

[0060] Furthermore, after receiving the second image and the target category, the server 11 may determine a target feature extraction model according to the target category, where the target feature extraction model is a feature extraction model associated with the target category.

[0061] Specifically, the server end 11 may be configured with one or more feature extraction models. The feature extraction model may be obtained by pre-training a preset model using training data. The feature extraction model may be used to extract feature information of an image.

[0062] In the first specific embodiment, the server 11 is configured with a feature extraction model, and the feature extraction model has an association relationship with each product category. In other words, there can be a one-to-many correspondence between the feature extraction model and the product category.

[0063] Specifically, the sample images used to train the feature extraction model include sample images corresponding to various product categories. Therefore, the feature extraction model can be used to extract features from images of various product categories.

[0064] It should be noted that, in the embodiment of the present invention, the sample images corresponding to each product category refer to sample images including images of products belonging to the product category.

[0065] In the second specific embodiment, the server 11 is configured with multiple feature extraction models, and the feature extraction models correspond to product categories one by one, that is, each feature extraction model has an association relationship with its uniquely corresponding product category.

[0066] Specifically, the training data for each feature extraction model includes sample images corresponding to the product category uniquely associated with that feature extraction model. Therefore, each feature extraction model can be used to extract features from images of its corresponding product category. This approach allows for the training of a corresponding feature extraction model for each product category, ensuring accurate product recognition.

[0067] In a third specific embodiment, the server 11 is configured with multiple feature extraction models. These multiple feature extraction models may include a general feature extraction model and at least one dedicated feature extraction model. In other words, each feature extraction model may be a dedicated feature extraction model or a general feature extraction model. More specifically, the number of general feature extraction models may be one, while the number of dedicated feature extraction models is not limited in this embodiment and may be determined based on actual circumstances.

[0068] Accordingly, the plurality of product categories may be divided into a first category group or a second category group according to the number of corresponding sample images. For example, the target category may belong to the first category group or the second category group.

[0069] More specifically, the first category group may include at least one first product category, wherein the number of sample images corresponding to each first product category is greater than a first preset threshold; and the second category group may include multiple second product categories, wherein the number of sample images corresponding to each second product category is less than or equal to the first preset threshold. The first preset threshold may be determined by the user based on actual circumstances.

[0070] For example, if the number of sample images corresponding to the target category is greater than a first preset threshold, the target category belongs to the first category group; otherwise, the target category belongs to the second category group.

[0071] On the one hand, the first product categories and the exclusive feature extraction models are in one-to-one correspondence, that is, the number of first product categories and exclusive feature extraction models can be the same, and each first product category uniquely corresponds to an exclusive feature extraction model.

[0072] More specifically, each dedicated feature extraction model is trained using sample images corresponding to its corresponding first product category as training data. Thus, each dedicated feature extraction model can be used to extract features from images of its corresponding product category. Therefore, each first product category is associated with its corresponding dedicated feature extraction model.

[0073] On the other hand, there is a many-to-one correspondence between the second product categories and the universal feature extraction model. That is, the multiple second product categories in the second category group all correspond to the universal feature extraction model. In other words, the multiple second product categories in the second category group all have an associated relationship with the universal feature extraction model.

[0074] In the application scenario where the product identification system 1 is used to identify the SKU to which accessories belong, the first category group may include, but is not limited to, rings, necklaces, and bracelets. The second category group may include, but is not limited to, brooches, earrings, and headwear.

[0075] By adopting such a solution, the accuracy of product recognition can be improved as much as possible when the number of sample images corresponding to product categories is unbalanced.

[0076] Specifically, among multiple product categories, there may be a part of product categories with a small number of registered products, resulting in a small number of sample images for this part of product categories. For example, in the application scenario of identifying accessories, the number of sample images of brooches, earrings and headwear is small compared to the number of sample images of rings, necklaces and bracelets. In such a case, if a feature extraction model is trained for each product category, it may cause inaccurate recognition results for both the first product category and the second product category. If a unique feature extraction model is trained for each product category, then for the second product category, there are fewer sample images and the requirements for model training and accuracy cannot be met. For this reason, in the solution of the present invention, unique and corresponding exclusive feature extraction models are trained for product categories with a large number of sample images, while a universal feature extraction model is trained for product categories with a small number of sample images. Adopting such a solution can maximize the accuracy of product recognition.

[0077] Based on the above, the server 11 can determine the target feature extraction model associated with the target category.

[0078] The following is a non-limiting description of the training process of the feature extraction model.

[0079] For each product category, the server 11 may obtain images of registered products belonging to the product category, which are recorded as sample images corresponding to the product category. The sample images may have pre-labeled tags, which may be used to indicate the SKU to which the registered products in the sample images belong.

[0080] In a specific implementation, the sample image can be uploaded by the client 12. More specifically, the registered product can be photographed by the photographing device 13 and then sent to the client 12. After the client 12 receives the sample image, it can pre-process the sample image and the user can label the sample image.

[0081] The pre-processing of the sample image by the client 12 may include: performing data augmentation processing on the sample image. The data augmentation processing may be any existing data augmentation processing method that can be used for image data augmentation processing, and this embodiment does not limit this.

[0082] Furthermore, for each feature extraction model, sample images corresponding to the product category with which it is associated can be used as training data for training to obtain a feature extraction model for extracting features from the image. This embodiment does not limit the training method, and various existing appropriate training methods can be used.

[0083] The product identification process of the server side 11 will be described below.

[0084] After receiving the second image sent by the client 12, the server 11 may input the second image into the target feature extraction model to obtain first feature information output by the target feature extraction model. The first feature information may be a feature vector, for example, a 512-dimensional feature vector or a 1024-dimensional feature vector. The dimension of the feature vector may be determined by the structure of the feature extraction model, and this embodiment does not impose any restrictions on the structure of the feature extraction model.

[0085] In a specific example, before inputting the second image into the target feature extraction model, the server 11 may preprocess the second image. Specifically, the server 12 may standardize the image. More specifically, the image may be standardized according to a preset size, where the preset size may be determined based on the target category. More specifically, the target feature extraction model may be determined based on the target category. Since the input image size of different target feature extraction models may be different, the preset size used in the standardization process may be determined based on the target feature extraction model.

[0086] Furthermore, the server 11 may determine a recognition result of the first image according to the first feature information.

[0087] Specifically, the server 11 may include a database module, which may store a plurality of registration feature information, wherein the registration feature information refers to feature information of the SKU, and the feature information of the SKU may be calculated based on a feature vector associated with the SKU.

[0088] More specifically, for each SKU, the feature vector associated with the SKU is the feature vector obtained by extracting features from the sample images associated with the SKU using the feature extraction model associated with the SKU. The feature extraction model associated with the SKU is a feature extraction model associated with the product category to which the SKU belongs, and the sample images associated with the SKU are sample images containing registered products belonging to the SKU.

[0089] In a specific example, the characteristic information of the SKU may be the average value of the characteristic vectors associated with the SKU.

[0090] Furthermore, a plurality of registered feature information may be read from the database module, and then similarities between the first feature information and each piece of registered feature information may be calculated, and the recognition result may be determined according to the similarities between the first feature information and each piece of registered feature information.

[0091] For example, the SKUs corresponding to a preset number of registered feature vectors with the highest similarity can be used as the recognition result, or the SKUs corresponding to registered feature information with similarity greater than a preset similarity threshold can be used as the recognition result. This embodiment does not limit the method for calculating similarity; for example, cosine similarity can be used.

[0092] In a non-limiting example, the database module may include multiple storage units, and the storage units correspond one to one with the product categories. As described above, each product category includes multiple SKUs. Therefore, each storage unit can be used to store the characteristic information of the SKU of the product category corresponding thereto. When determining the recognition result, the server end 11 can read the characteristic information of multiple SKUs (that is, multiple registered characteristic information) from the storage unit corresponding to the target category, and determine the recognition result based on the characteristic information of the multiple SKUs read. By adopting such a solution, the characteristic information that needs to be read each time can be reduced, and the server end can be avoided from stalling, which is conducive to improving the efficiency of product recognition.

[0093] Thus, the server 11 can obtain the identification result of the product to be identified, and the server 11 can send the identification result to the client 12, and the client 12 can display the received identification result through the user interface.

[0094] During the use of the product identification system 1, one or more clients 12 may also register a new SKU, wherein the process of registering a SKU is the process of determining the characteristic information of the SKU.

[0095] Specifically, the user can use the camera 13 to shoot the newly added product to obtain a second sample image, and upload the second sample image to the server 11 through the client 12. The second sample image also has a pre-marked label, and the SKU to which the newly added product belongs is the newly added SKU.

[0096] Furthermore, server 11 can input the second sample image into a feature extraction model associated with the newly added SKU to obtain a feature vector for the second sample image. Furthermore, feature information for each newly added SKU can be calculated based on the feature vector of the second sample image. For ease of description, the feature information of the historical SKUs will be referred to as the first registered feature information, and the feature information of the newly added SKUs will be referred to as the second registered feature information.

[0097] In a specific implementation, there are multiple clients 12, and the database module may include a permanent storage module and a temporary storage module. The permanent storage module is used to store the first registration feature information. If one of the multiple clients 12 simultaneously uses the product identification system to identify products and register new SKUs, the second registration feature information may be stored in the temporary storage module. After identification is complete, the second registration feature information may be written to the permanent storage module, and the second registration feature information in the temporary and permanent storage modules may be deleted.

[0098] More specifically, when the second registration feature information is stored in the temporary storage module, during the identification process, the first registration feature information can be read from the permanent storage module, and the first identification result can be determined based on the first feature information and the first registration feature information. If the first identification result meets the preset conditions, the first identification result is used as the identification result; if the first identification result does not meet the preset conditions, the second registration feature information is read from the temporary storage module, and the second identification result is determined based on the first feature information and the second registration feature information, and then the identification result is determined based on the first identification result and the second identification result. Among them, the preset conditions may include one or more of the following: the number of SKUs determined is greater than the preset number threshold, and the similarity of the SKUs determined is greater than the preset similarity threshold. In other words, the identification result of the product to be identified can be determined in the historical SKU first. If a certain number or a higher accuracy identification result cannot be determined in the historical SKU, the identification result of the product to be identified is determined in combination with the newly added SKU.

[0099] In a non-limiting example, when the client 12 pre-processes the first image, if it is determined that the product to be identified spans multiple placement zones (for example, the placement position of the product to be identified in the camera is partially deviated, resulting in it occupying multiple placement zones), multiple target categories are determined. In this case, the server 11 selects multiple target feature extraction models and obtains multiple first feature information. In this case, the client 12 can determine the weights of the multiple target categories based on the proportion of the product to be identified falling into each placement zone. For example, if 80% of the product to be identified falls into placement zone A and 20% of the product to be identified falls into placement zone B, then the weight of product category a corresponding to placement zone A is 80%, and the weight of product category b corresponding to placement zone B is 20%. Accordingly, the server 11 can determine the weight of each first feature information based on the weight of each target category, and calculate (for example, weighted average) the final first feature information based on each first feature information and its weight, and then use the final first feature information to determine the recognition result. In this way, even when the product to be identified is deviated and spans multiple placement zones, the feature information can be determined relatively accurately to perform product recognition.

[0100] In another non-limiting example, when pre-processing the first image, client 12 identifies that the product to be identified spans multiple placement zones, and then determines the product category corresponding to the placement zone with the largest percentage of products falling within it as the target category, while ignoring the product categories corresponding to the other placement zones. For example, if 80% of the product to be identified falls within placement zone A and 20% falls within placement zone B, then the product category corresponding to placement zone A may be determined as the target category.

[0101] In another non-limiting example, when the client 12 pre-processes the first image, in addition to determining the target category, it can also determine the candidate category based on the placement area where the product to be identified is located. The candidate category is the product category corresponding to the adjacent area, and the adjacent area is the placement area adjacent to the placement area where the product to be identified is located; in addition to inputting at least a portion of the first image into the target feature extraction model, the server 11 also inputs it into the candidate feature extraction model associated with the candidate category, and calculates (for example, takes the average) to obtain the first feature information based on the results output by the target feature extraction model and each candidate feature extraction model, and then determines the recognition result based on the first feature information. Using such a solution can improve the fault tolerance rate. This is because when the user places the product to be identified, he may mistakenly place the product to be identified in the wrong place. In practice, it is usually placed in a placement area adjacent to the correct placement area. Using the above solution, the feature information can be determined by using the actual placement area and its surrounding adjacent placement areas at the same time, so that this situation can be taken into account.

[0102] In the solution of the embodiment of the present invention, the server 12 can also perform incremental training on the feature extraction model.

[0103] Specifically, the server 11 may use the second sample image and at least a portion of the first sample image as training data to train the feature extraction model to update the feature extraction model. The second sample image may have been previously uploaded to the server 11 by the client 12 and stored in the database module. For more information about the second sample image, please refer to the above description and will not be repeated here.

[0104] More specifically, when incrementally training each feature extraction model, the training data used for incremental training are all sample images associated with the feature extraction model. The association between the feature extraction model and the sample images may mean that the feature extraction model and the sample images correspond to the same product category.

[0105] In the scheme of the embodiment of the present invention, when performing incremental training, the second sample image and a portion of the first sample image are used as training data, which is conducive to making the updated feature extraction model have better expressive ability on both new and old training data. Compared with the scheme of only using the second sample image for incremental training, the scheme of the embodiment of the present invention can continuously optimize the performance of the feature extraction model, thereby improving the accuracy of product recognition.

[0106] In a specific embodiment, the backend administrator can pre-set the triggering conditions for incremental training, and when the triggering conditions are triggered, the feature extraction model is incrementally trained. It should be noted that the backend administrator can be the user of the server 11, and the user is the user of the client 12.

[0107] Specifically, the server end 11 can determine whether the preset incremental training conditions are met according to a preset time interval. When the preset incremental training conditions are met, the feature extraction model can be incrementally trained. Among them, the incremental training conditions may include one or more of the following: the system time reaches the preset training time, the number of second sample images reaches the second preset threshold. The preset training time and the second preset threshold can be preset. For example, when the number of second sample images uploaded by the client 12 reaches the second preset threshold, and the system time reaches the preset training time, the feature extraction model can be incrementally trained. For another example, the preset training time can be from 0:00 to 5:00 every day, so as not to affect the normal use of the product identification system 1.

[0108] In a specific example, the feature extraction model may be incrementally trained if the background administrator pre-sets the incremental training conditions. That is, if the background administrator does not set the incremental training conditions, the feature extraction model may not be incrementally trained.

[0109] In a non-limiting embodiment, if the preset incremental training conditions are met, it can be determined whether the incremental training function is enabled. If so, incremental training can continue. If the incremental training function is not enabled, incremental training will not be performed until it is detected that the incremental training function is enabled. The incremental training function can be enabled by a backend administrator. When the user enables the incremental training function, the incremental training function is enabled. When the backend administrator disables the incremental training function, the incremental training function is disabled. This solution can further reduce the conflict between the use and maintenance of the product identification system.

[0110] Reference Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of the relationship between modules in a product identification system according to an embodiment of the present invention. Figure 4 Schematic diagram of the architecture of a product identification system in an embodiment of the present invention.

[0111] like Figure 3 As shown, the product identification system may include: a storage module 31, a management module 32, a call module 33, an algorithm module 34, a task scheduling module 35, and a graphical user interface (GUI) module 36. The GUI module 36 is deployed on the client side, while the storage module 31, the management module 32, the call module 33, the algorithm module 34, and the task scheduling module 35 are deployed on the server side.

[0112] The storage module 31 may include a database module and a local storage module (e.g., memory). In a specific example, the database module may include a Redis database, a MySQL database, and a MongoDB database. The MySQL database may be used to store structured data, while the MongoDB database and the local storage module may be used to store unstructured data. The MySQL and MongoDB databases may be permanent storage, while the Redis database may be temporary storage.

[0113] In specific implementations, the MySQL database can be used to store one or more of the following: ① user information; ② feature extraction model parameter configuration; ③ access logs of various terminal interface calls and user logins; and ④ task scheduling parameter configuration. The MongoDB database can be used to store one or more of the following: ① feature information calculated by the feature extraction model; ② sample images and their labels. The local storage module can be used to store the feature extraction model.

[0114] The management module 32 manages the hardware and software of the product identification system. Specifically, it manages interface permissions, task scheduling, and logging during the use of the product identification system. Furthermore, the management module supports backend administrator permissions, model parameter configuration and modification, and user authentication.

[0115] The calling module 33 can be used to provide an interface to the client to enable data interaction between the client and the server. More specifically, the calling module can be connected to the GUI module. For example, the interface provided by the calling module can be a RESTFul API that supports GET and POST request types.

[0116] The algorithm module 34 can be directly connected to the storage module 31, and is associated with the calling module 33 through the storage module 31. The algorithm module 34 can be used to process the image and obtain the recognition result based on the image. The algorithm module can be used for one or more of the following: ① Image preprocessing, including but not limited to format conversion, image cropping, image enhancement, to provide preparatory work for model training / model expression capabilities; ② Feature extraction based on the feature extraction model, wherein the feature extraction model can be a neural network such as ResNet-50, AlexNet, VGG, etc.; ③ Training module, including initial training of the feature extraction model, configuration of model parameters, etc.; ④ Maintenance and update of the model, maintenance of later training data, and model training after manual correction of labels.

[0117] The task calling module 35 can be used to provide an asynchronous task function to avoid the GUI module 36 being in a long wait during the calculation process of the algorithm model 34, which results in a poor user experience. The asynchronous task function can be implemented through Celery.

[0118] The GUI module 36 can be used to interact with the user, and can be an exe software that can be directly run on the client, and can be developed using PyQT5 technology.

[0119] The following is a non-limiting description of the process of building a product identification system.

[0120] On a server with a Linux operating system, install Python, MySQL, MongoDB, Redis, and Nginx software, and complete the configuration of the MySQL, MongoDB, and Redis databases.

[0121] Use the Django framework django-admin command of the Python programming language to create a new product identification system project, and develop an Application through the command django-admin manage.py startapp. In the settings.py file of the project, set the MySQL and MongoDB database connection configurations, and modify the settings.py file to support multiple database links. Design, develop, and create data tables and table fields according to the detailed requirements of the product identification system. The management of database tables is uniformly implemented using the models function of the Django framework. Use the user authentication module that comes with the Django framework, and conduct secondary development on the log module and task scheduling module. Use Celery and Redis for configuration, and configure them in the settings.py file of the Django framework to implement asynchronous task functions.

[0122] In addition, we've developed image dataset management functionality within the Django admin interface, supporting data addition, deletion, and modification, facilitating the management of training data. Beyond daily data management and model training parameter configuration, the admin interface also manages operator permissions, allowing qualified accounts to perform operations such as addition, deletion, and modification directly through the interface, as well as configure tasks within the task scheduling module.

[0123] In the solution of the embodiment of the present invention, the implementation of the product identification system can all adopt the Python programming language, which can effectively avoid unnecessary compatibility issues. In the specific implementation, OpenCV and PyTorch modules can be used to preprocess the image and build the feature extraction model. The training set data is stored in the MongoDB database together with the label after image preprocessing, and the model of the feature extraction model obtained after the neural network training (including the network structure and parameters) is stored in the local storage module of the server. In addition, the incremental training of the feature extraction model is supported by the task scheduling module 35. Under the management of the management module 32, the task scheduling module 35 manages the model by updating, deleting, testing, etc. through conditional judgment or scheduled tasks.

[0124] Reference Figure 5 , Figure 5 This is a flow chart of a product identification method according to an embodiment of the present invention. Figure 5 The use process of the above-mentioned product identification system is described in a non-limiting manner. Figure 5 The illustrated method may include steps S51 to S58 .

[0125] Step S51: The GUI module pre-processes the first image to obtain a target category.

[0126] In a specific implementation, the user operates the exe program of the GUI module and directly clicks the [Recognition] button with the mouse to pre-process the first image. The GUI module can encapsulate image target detection algorithms, image quality diagnosis algorithms, etc.

[0127] Step S52: The GUI module sends a recognition request to the calling module. Correspondingly, the calling module receives the recognition request.

[0128] In a specific implementation, the GUI module sends a recognition request by calling an interface provided by the module, wherein the recognition request includes the second image.

[0129] Step S53: The calling module calls the task scheduling module.

[0130] In a specific implementation, the calling module may send the identification request to the task scheduling module.

[0131] Step S54: The task scheduling module sends a task identification code to the calling module and writes the second image into the storage module. The task identification code is used to indicate that the recognition request has been successfully received and can be used to uniquely identify the recognition request.

[0132] Step S55: The calling module sends the task identification code to the GUI module.

[0133] Step S56: the algorithm module processes the second image to obtain a recognition result, and stores the recognition result in the Redis database.

[0134] Step S57: The calling module receives the query request sent by the GUI module, wherein the query request includes the task identification code.

[0135] Step S58: The calling module reads the recognition result from the Redis database and sends the recognition result to the GUI module.

[0136] As described above, the solution provided by the embodiment of the present invention can integrate the image recognition algorithm with hardware to achieve the integration of software and hardware engineering, and fully utilize the advantages of the Django framework. A RESTFul API interface is used to implement request responses across the server and client, as well as incremental autonomous learning and training of the model, thereby constructing a reasonable, standardized and scientific product recognition system.

[0137] It should be understood that the term "and / or" as used herein simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " as used herein indicates that the related objects are in an "or" relationship.

[0138] The "multiple" appearing in the embodiments of the present application refers to two or more. The first, second, etc. descriptions appearing in the embodiments of the present application are only for illustration and distinction of the description objects. There is no order, nor does it represent a special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application. Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

[0139] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A product identification system, characterized in that: The system comprises: a photographing device, the photographing device comprising a placement portion and a camera, the placement portion being used to place the product to be identified and having a plurality of placement areas, the camera being used to photograph the placement portion to obtain a first image; The client is configured to pre-process the first image and receive a recognition result, wherein the pre-processing includes: determining a product category of the product to be identified based on a placement area where the product to be identified is located, recording the category as a target category, and wherein the placement area corresponds to the product category in a one-to-one manner; The server is configured to select a target feature extraction model associated with the target category from a plurality of feature extraction models, input at least a portion of the first image into the target feature extraction model to obtain first feature information, determine a recognition result based on the first feature information, and send the recognition result to the client; The product category refers to the type of product. Each product category includes multiple SKUs. Different product categories include different SKUs. The recognition result is the SKU to which the product to be recognized belongs.

2. The product identification system according to claim 1, characterized in that: Each placement area has an indicator line and a visual label, wherein the indicator line is used to determine the range of the placement area, and the visual label is used to indicate the product category corresponding to the placement area.

3. The product identification system according to claim 1, characterized in that: The photographing device further comprises: a placement box, wherein the placement portion and the camera are located inside the placement box; Wherein, at least a portion of the placement box is opaque.

4. The product identification system according to claim 3, characterized in that: At least a portion of the placement box is white.

5. The product identification system according to claim 3, characterized in that: The shooting device further includes: a lighting device, and the lighting device is located inside the placement box.

6. The product identification system according to claim 1, characterized in that: The target category belongs to a first category group or a second category group, wherein the first category group includes at least one first product category, and the number of sample images of each first product category is greater than a first preset threshold; the second category group includes multiple second product categories, and the number of sample images corresponding to each second product category is less than or equal to the first preset threshold; The multiple feature extraction models include: a general feature extraction model and at least one exclusive feature extraction model, wherein the general feature extraction model is trained using sample images corresponding to the multiple second product categories as training data, and the exclusive feature extraction models correspond one-to-one to the first product categories, and each exclusive feature extraction model is trained using sample images corresponding to the first product category corresponding to it as training data.

7. The product identification system according to claim 6, characterized in that: The at least one first product category includes one or more of the following: rings, necklaces, and bracelets; the multiple second product categories include: brooches, earrings, and headwear.

8. The product identification system according to claim 7, characterized in that: Preprocessing the first image includes: If the target category is necklaces, image detection is performed on the first image to obtain a detection result, where the detection result is used to indicate whether the product to be identified is placed in accordance with preset requirements; If the detection result indicates that the product to be identified is not placed according to the preset requirements, the user is reminded to adjust the placement of the product to be identified.

9. The product identification system according to claim 1, characterized in that: The client is further configured to obtain a second sample image and send the second sample image to the server; The server is further configured to use at least a portion of the first sample image and the second sample image as training data to train the feature extraction model, so as to update the feature extraction model; The first sample image and the second sample image both have pre-labeled labels, and the labels are used to indicate the SKU to which the product in the sample image belongs. The first sample image is a sample image used to train the feature extraction model before the update.

10. The product identification system according to claim 9, characterized in that: Before training the feature extraction model using at least a portion of the first sample image and the second sample image as training data, the server is further configured to: determining whether a preset incremental training condition is satisfied, and if so, using the second sample image and at least a portion of the first sample image as training data to train the feature extraction model; The incremental training conditions include one or more of the following: The system time reaches the preset training time, and the number of the second sample images reaches a second preset threshold.

11. The product identification system according to claim 1, wherein: The system further includes: a database module, the database module including a plurality of storage units, the storage units corresponding to product categories one by one, and each storage unit being used to store feature information of SKUs belonging to the corresponding product category.

12. The product identification system according to claim 11, characterized in that: The feature information of each SKU is calculated based on a feature vector associated with the SKU, wherein the feature vector associated with the SKU is obtained by performing feature extraction on a sample image containing products belonging to the SKU using the feature extraction model.

13. The product identification system according to claim 1, wherein: Before inputting at least a portion of the first image into the target feature extraction model, the server is further configured to perform normalization processing on the received image according to a preset size, wherein the preset size is determined according to the target category.

14. The product identification system according to claim 1, wherein: The server side includes: A calling module is used to provide an interface to the client to implement data interaction between the client and the server.

Images