Product identification method and device, computer equipment and storage medium

By using feature recognition and matching models, the problems of low efficiency and high cost of existing product authentication methods have been solved, achieving efficient and accurate product authenticity authentication.

CN119049020APending Publication Date: 2024-11-29巽风科技(贵州)有限公司

Patent Information

Application Number
CN202410894811.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing methods for authenticating products rely on human experience, which is inefficient and costly. Furthermore, automated methods require complex modifications to printing and packaging processes, making them difficult to adapt to industrial applications.

Method used

The feature recognition model identifies the feature regions of the product appearance image, performs local and global feature extraction, constructs a combined feature vector, and uses a feature matching model for comparison, outputting the comparison results and confidence level, and combining the database for identification.

Benefits of technology

It enables efficient and accurate product authenticity verification, improves verification efficiency and result reliability, and reduces labor costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049020A_ABST
    Figure CN119049020A_ABST
Patent Text Reader

Abstract

The invention relates to a product identification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an appearance image of a to-be-identified product, and identifying a first feature region and first data information through a feature identification model; performing local feature extraction and global feature extraction on the first feature region through a feature extraction model to obtain a first local feature vector and a first global feature vector; splicing the first local feature vector and the first global feature vector into a first combined feature vector; based on the first data information, searching a second combined feature vector in a first database; inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result; responding to the comparison result that the product identification fails, and returning a prompt that the product identification fails; and returning the first data information in response to a successful comparison result. By adopting the method, the authenticity of the product can be accurately and efficiently identified.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a product identification method, device, computer equipment and storage medium. Background Art

[0002] The phenomenon of product counterfeiting can cause serious harm to both manufacturers and consumers. On the one hand, counterfeit products may harm the health of consumers, and on the other hand, counterfeit products can cause great damage to the manufacturer's brand. Therefore, manufacturers often invest a lot of manpower and material resources to authenticate the authenticity of products. In the authentication of product authenticity, authenticating the product by product appearance is a very important means.

[0003] The existing product authenticity identification is mainly carried out manually by professional certified staff. Manual identification relies heavily on the staff's experience, with low identification efficiency and high labor costs. The automatic anti-counterfeiting method usually requires the modification of the printing and packaging process, which is relatively complicated. In summary, the existing product authentication method is costly and difficult to adapt to existing industrial application scenarios.

[0004] Therefore, there is an urgent need to propose a product identification method, device, computer equipment and storage medium that can accurately and efficiently identify the authenticity of products. Summary of the invention

[0005] Based on this, it is necessary to provide a product identification method, device, computer equipment and storage medium that can accurately and efficiently identify the authenticity of products in response to the above technical problems.

[0006] In a first aspect, a product identification method is provided, the method comprising:

[0007] Acquire an appearance image of a product to be identified, and identify a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model;

[0008] Performing local feature extraction and global feature extraction on the first feature region by using a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0009] concatenating the first local feature vector and the first global feature vector into a first combined feature vector;

[0010] Based on the first data information, searching for a second combined feature vector of the product to be identified in the first database;

[0011] Inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence;

[0012] In response to the comparison result being a failure, returning a prompt indicating that the product identification has failed;

[0013] In response to a successful comparison result, the first data information is returned, and the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence are stored in the first database.

[0014] In one embodiment, before obtaining the appearance image of the product to be identified, the method further includes:

[0015] Acquire an appearance image of a product that has not been shipped out of the factory, and identify a second feature region of the appearance image of the product that has not been shipped out of the factory and second data information of the second feature region through the feature recognition model, wherein the product that has not been shipped out of the factory refers to a product to be identified that has not been shipped out of the factory, and the second feature region is the same as the first feature region;

[0016] Performing local feature extraction and global feature extraction on the second feature region by using the feature extraction model to obtain a second local feature vector and a second global feature vector;

[0017] concatenating the second local feature vector and the second global feature vector into a second combined feature vector;

[0018] An association relationship between the second data information and the second combined feature vector is constructed, and based on the association relationship, the second data information and the second combined feature vector are stored in the first database.

[0019] In one embodiment, the step of acquiring an appearance image of the product to be identified, and identifying a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model includes:

[0020] According to a preset area size, a first feature area of ​​the appearance image of the product to be identified is identified by a text detection algorithm;

[0021] The first data information is identified by a target detection algorithm for a first feature area of ​​the appearance image of the product to be identified, wherein the first data information includes the production year, production batch and production number of the product to be identified.

[0022] In one embodiment, extracting local features from the first feature region using a feature extraction model to obtain a first local feature vector includes:

[0023] Establishing a Gaussian pyramid based on the first feature area, generating a differential pyramid based on the Gaussian pyramid, and determining coordinates of potential feature points in the differential pyramid;

[0024] Based on the coordinates of the potential feature points, the feature points are fitted by Taylor expansion, and the coordinates of the actual feature points are obtained according to the fitting results, and the edge feature points in the actual feature points are removed by a corner point detection algorithm to obtain n final feature points, where n is a positive integer;

[0025] Acquire the scale value of the final feature point, and perform finite difference on the final feature point based on the scale value to obtain the amplitude and argument of the pixel points in the area centered on the final feature point;

[0026] The region centered on the final feature point is divided into 4*4 sub-regions, and the gradient histograms in 8 directions in each sub-region are calculated to generate a first local feature vector with n*128 dimensions.

[0027] In one embodiment, performing global feature extraction on the first feature region by using a feature extraction model to obtain a first global feature vector includes:

[0028] Based on the first feature area, obtaining a first coding image;

[0029] Extracting features of the first coding image through a residual network;

[0030] The features of the first coding image are mapped into a first global feature vector of 128 dimensions through regularization.

[0031] In one embodiment, the step of concatenating the first local feature vector and the first global feature vector into a first combined feature vector includes:

[0032] Concatenate the first local feature vector of n*128 dimensions with the first global feature vector of 128 dimensions to obtain a first combined feature vector of (n+1)*128 dimensions;

[0033] The step of concatenating the second local feature vector and the second global feature vector into a second combined feature vector comprises:

[0034] The second local feature vector of n*128 dimensions is concatenated with the second global feature vector of 128 dimensions to obtain a second combined feature vector of (n+1)*128 dimensions;

[0035] The step of inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence level includes:

[0036] The first combined feature vector of (n+1)*128 dimensions is concatenated with the second combined feature vector of (n+1)*128 dimensions to obtain a combined feature vector pair of (n+1)*256 dimensions;

[0037] The combined feature vector of the (n+1)*256 dimensions is compared with the input feature matching model, and the comparison result and comparison confidence are output.

[0038] In one embodiment, the feature matching model is trained in the following manner:

[0039] Performing data enhancement on the first coding image to obtain a second coding image;

[0040] Obtain a third coding picture in the second database, and construct a coding picture triplet based on the first coding picture, the second coding picture and the third coding picture, wherein the third coding picture is any picture in the second database that is unrelated to the first coding picture;

[0041] Extracting features of the second coding image and the third coding image respectively through the residual network;

[0042] Mapping the features of the second coding image and the third coding image into global feature vectors of 128 dimensions respectively through regularization;

[0043] Generate local feature vectors of the second coding image and the third coding image respectively, and concatenate the local feature vectors of the second coding image and the third coding image with the global feature vector into a combined feature vector of (n+1)*128 dimensions;

[0044] The combined feature vector of the first coding image is concatenated with the combined feature vector of the second coding image to obtain a first combined feature vector pair, and the first combined feature vector pair is input into a feature matching model for comparison to obtain the similarity between the first coding image and the second coding image;

[0045] The combined feature vector of the first coding image is concatenated with the combined feature vector of the third coding image to obtain a second combined feature vector pair, and the second combined feature vector pair is input into a feature matching model for comparison to obtain the similarity between the first coding image and the third coding image;

[0046] The parameters of the feature matching model are adjusted until the similarity between the first coding image and the second coding image is close to 1, and the similarity between the first coding image and the third coding image is close to 0.

[0047] In a second aspect, a product identification device is provided, the device comprising:

[0048] An acquisition module, the acquisition module is used to acquire an appearance image of the product to be identified;

[0049] An identification module, the identification module is used to identify a first feature area of ​​the appearance image of the product to be identified and first data information of the first feature area through a feature recognition model;

[0050] An extraction module, the extraction module is used to perform local feature extraction and global feature extraction on the first feature area through a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0051] A splicing module, configured to splice the first local feature vector and the first global feature vector into a first combined feature vector;

[0052] A search module, the search module is used to search the second combined feature vector of the product to be identified in the first database based on the first data information;

[0053] A comparison module, wherein the comparison module is used to input the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and output a comparison result and a comparison confidence; in response to a comparison result of failure, return a prompt of product identification failure; in response to a comparison result of success, return the first data information, and store the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence in the first database.

[0054] In a third aspect, a computer device is provided, comprising one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, execute the steps of the method described in any one of the first aspects above.

[0055] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the first aspects are performed.

[0056] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any one of the above-mentioned first aspects are implemented.

[0057] The above-mentioned product identification method, device, computer equipment and storage medium identify the first feature area of ​​the appearance image of the product to be identified and the first data information of the first feature area through a feature recognition model; perform local feature extraction and global feature extraction on the first feature area through a feature extraction model, and splice the first local feature vector and the first global feature vector into a first combined feature vector; based on the first data information, search for the second combined feature vector of the product to be identified in the first database; input the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and output the comparison result and comparison confidence, so as to improve the product identification efficiency and the credibility of the identification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a product identification method in one embodiment;

[0059] Figure 2 is a schematic diagram of a flow chart of feature point detection in one embodiment;

[0060] Figure 3 A schematic diagram of using a feature matching model for comparison in one embodiment;

[0061] Figure 4 is a structural block diagram of a product identification device in one embodiment;

[0062] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] Embodiment 1

[0065] In one embodiment, Figure 1 As shown, a product identification method is provided, the method comprising:

[0066] Acquire an appearance image of a product to be identified, and identify a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model;

[0067] Performing local feature extraction and global feature extraction on the first feature region by using a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0068] concatenating the first local feature vector and the first global feature vector into a first combined feature vector;

[0069] Based on the first data information, searching for a second combined feature vector of the product to be identified in the first database;

[0070] Inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence;

[0071] In response to the comparison result being a failure, returning a prompt indicating that the product identification has failed;

[0072] In response to a successful comparison result, the first data information is returned, and the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence are stored in the first database.

[0073] Specifically, a first feature area of ​​the appearance image of the product to be identified and first data information of the first feature area are identified through a feature recognition model; local feature extraction and global feature extraction are performed on the first feature area through a feature extraction model, and the first local feature vector and the first global feature vector are spliced ​​into a first combined feature vector; based on the first data information, a second combined feature vector of the product to be identified in a first database is searched; the first combined feature vector and the second combined feature vector are input into a feature matching model for comparison, and the comparison result and comparison confidence are output, so as to improve the product identification efficiency and the credibility of the identification results.

[0074] Specifically, the first database includes but is not limited to a blockchain storage system, and the user can query detailed information on the comparison result through the hash on the chain.

[0075] In one embodiment, before obtaining the appearance image of the product to be identified, the method further includes:

[0076] Acquire an appearance image of a product that has not been shipped out of the factory, and identify a second feature region of the appearance image of the product that has not been shipped out of the factory and second data information of the second feature region through the feature recognition model, wherein the product that has not been shipped out of the factory refers to a product to be identified that has not been shipped out of the factory, and the second feature region is the same as the first feature region;

[0077] Performing local feature extraction and global feature extraction on the second feature region by using the feature extraction model to obtain a second local feature vector and a second global feature vector;

[0078] concatenating the second local feature vector and the second global feature vector into a second combined feature vector;

[0079] An association relationship between the second data information and the second combined feature vector is constructed, and based on the association relationship, the second data information and the second combined feature vector are stored in the first database.

[0080] Specifically, the authenticity of the product is determined by collecting product appearance images at different stages, that is, feature vectors are extracted from feature areas before leaving the factory, and the extracted feature vectors are used for product identification after leaving the factory.

[0081] Specifically, when users need to identify the authenticity of a product, they need to collect images of the same location. In order to improve the success rate of the appearance images of the product to be identified, it is necessary to perform real-time photo quality detection when the user collects the image to improve the user experience. Generally, the appearance image needs to have 200 feature points.

[0082] In one embodiment, the step of acquiring an appearance image of the product to be identified, and identifying a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model includes:

[0083] According to a preset area size, a first feature area of ​​the appearance image of the product to be identified is identified by a text detection algorithm;

[0084] The first data information is identified by a target detection algorithm for a first feature area of ​​the appearance image of the product to be identified, wherein the first data information includes the production year, production batch and production number of the product to be identified.

[0085] Specifically, the region of interest (ROI) extraction algorithm DBNet++ is used to extract a 640*320 ROI area; and the data information of the ROI area is identified by the RCNN algorithm.

[0086] In one embodiment, extracting local features from the first feature region using a feature extraction model to obtain a first local feature vector includes:

[0087] Establishing a Gaussian pyramid based on the first feature area, generating a differential pyramid based on the Gaussian pyramid, and determining coordinates of potential feature points in the differential pyramid;

[0088] Based on the coordinates of the potential feature points, the feature points are fitted by Taylor expansion, and the coordinates of the actual feature points are obtained according to the fitting results, and the edge feature points in the actual feature points are removed by a corner point detection algorithm to obtain n final feature points, where n is a positive integer;

[0089] Acquire the scale value of the final feature point, and perform finite difference on the final feature point based on the scale value to obtain the amplitude and argument of the pixel points in the area centered on the final feature point;

[0090] The region centered on the final feature point is divided into 4*4 sub-regions, and the gradient histograms in 8 directions in each sub-region are calculated to generate a first local feature vector with n*128 dimensions.

[0091] Specifically, generally more than 300 128-dimensional feature vectors are used to represent the first feature region.

[0092] In one embodiment, performing global feature extraction on the first feature region through a feature extraction model to obtain a first global feature vector includes:

[0093] Based on the first feature area, obtaining a first coding image;

[0094] Extracting features of the first coding image through a residual network;

[0095] The features of the first coding image are mapped into a first global feature vector of 128 dimensions through regularization.

[0096] Specifically, the coding picture is a picture taken by an industrial computer or a client.

[0097] In one embodiment, the step of concatenating the first local feature vector and the first global feature vector into a first combined feature vector includes:

[0098] Concatenate the first local feature vector of n*128 dimensions with the first global feature vector of 128 dimensions to obtain a first combined feature vector of (n+1)*128 dimensions;

[0099] The step of concatenating the second local feature vector and the second global feature vector into a second combined feature vector comprises:

[0100] The second local feature vector of n*128 dimensions is concatenated with the second global feature vector of 128 dimensions to obtain a second combined feature vector of (n+1)*128 dimensions;

[0101] The step of inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence level includes:

[0102] The first combined feature vector of (n+1)*128 dimensions is concatenated with the second combined feature vector of (n+1)*128 dimensions to obtain a combined feature vector pair of (n+1)*256 dimensions;

[0103] The combined feature vector of the (n+1)*256 dimensions is compared with the input feature matching model, and the comparison result and comparison confidence are output.

[0104] Specifically, the local feature vectors and the global feature vectors are concatenated to improve the robustness of the system and enhance the product identification efficiency and the credibility of the identification results.

[0105] In one embodiment, the feature matching model is trained in the following manner:

[0106] Performing data enhancement on the first coding image to obtain a second coding image;

[0107] Obtain a third coding picture in the second database, and construct a coding picture triplet based on the first coding picture, the second coding picture and the third coding picture, wherein the third coding picture is any picture in the second database that is unrelated to the first coding picture;

[0108] Extracting features of the second coding image and the third coding image respectively through the residual network;

[0109] Mapping the features of the second coding image and the third coding image into global feature vectors of 128 dimensions respectively through regularization;

[0110] Generate local feature vectors of the second coding image and the third coding image respectively, and concatenate the local feature vectors of the second coding image and the third coding image with the global feature vector into a combined feature vector of (n+1)*128 dimensions;

[0111] The combined feature vector of the first coding image is concatenated with the combined feature vector of the second coding image to obtain a first combined feature vector pair, and the first combined feature vector pair is input into a feature matching model for comparison to obtain the similarity between the first coding image and the second coding image;

[0112] The combined feature vector of the first coding image is concatenated with the combined feature vector of the third coding image to obtain a second combined feature vector pair, and the second combined feature vector pair is input into a feature matching model for comparison to obtain the similarity between the first coding image and the third coding image;

[0113] The parameters of the feature matching model are adjusted until the similarity between the first coding image and the second coding image is close to 1, and the similarity between the first coding image and the third coding image is close to 0.

[0114] Specifically, the output result of the feature matching model is made more accurate by using the coding image triples.

[0115] Specifically, the feature matching model includes a comparison network, which inputs the combined feature vector pair into the comparison network and finally outputs a binary value [a, 1-a], which represents the similarity between the coded images. When a>0.5, the similarity between the coded images is high and the comparison is successful; when a<0.5, the similarity between the coded images is low and the comparison fails.

[0116] Specifically, the steps of extracting local feature vectors through the feature extraction model are as follows:

[0117] 1. As Figure 2 As shown, feature point detection:

[0118] When performing extreme point detection, the specific steps are: first, a Gaussian pyramid is established for the selected area image, then a differential pyramid is obtained through the Gaussian pyramid, and finally, extreme points are obtained in the differential pyramid.

[0119] 1) The steps to establish the Gaussian pyramid are as follows:

[0120] (1) The original image is first doubled and then used as the first layer of the first group of Gaussian pyramids. The first layer of the first group of images is subjected to Gaussian convolution and then used as the second layer of the first group of pyramids. The parameter σ is set to a fixed value of 1.6 in the present invention;

[0121] (2) Multiply σ by a scaling factor k to get a new smoothing factor σ = k*σ, and use it to smooth the first group of second layer images. The resulting image is used as the third layer.

[0122] (3) Repeat this process until L layers of images are obtained. In the same group, the size of each layer of images is the same, but the smoothing coefficients are different. Their corresponding smoothing coefficients are: 0, σ, kσ, k2σ, k3σ...k (L-2) σ.

[0123] (4) The third-to-last layer image of the first group is downsampled with a scale factor of 2, and the obtained image is used as the first layer of the second group. Then, the first layer image of the second group is Gaussian smoothed with a smoothing factor of σ to obtain the second layer of the second group, just like in step (2). In this way, the L-layer images of the second group are obtained. The sizes of the images in the same group are the same, and the corresponding smoothing coefficients are: 0, σ, kσ, k2σ, k3σ...k (L-2) However, the second set of images is half the size of the first set of images.

[0124] (5) By repeatedly executing this process, we can obtain a total of O groups, each with L layers, totaling O*L images, which together constitute a Gaussian pyramid.

[0125] In the same group, the sizes of images in different layers are the same, and the Gaussian smoothing factor σ of the next layer of images is K times the smoothing factor of the previous layer of images. In different groups, the first image of the next group is half the sample of the third to last image of the previous group, and the image size is half of that of the previous group.

[0126] 2) Difference pyramid: The difference pyramid is obtained by subtracting two adjacent layers of the Gaussian pyramid.

[0127] 3) Spatial extreme point detection:

[0128] a. Definition of extreme points: When each pixel is compared with all its neighboring points, it is an extreme point when it is greater than (or less than) all its neighboring points in the image domain and scale domain.

[0129] b. In order to find extreme points, each pixel is compared with all its neighboring points to see whether it is larger or smaller than its neighboring points in the image domain and scale domain. Each detection point is compared with 8 neighboring points in the same layer and 9×2 points corresponding to the upper and lower adjacent layers, a total of 26 points, to ensure that extreme points are detected in both scale space and two-dimensional image space.

[0130] 2. Accurate positioning of feature points: The image data stored in the computer is discrete, and the feature points we found before are the extreme points in the discrete space, but the extreme points in the discrete space are not the extreme points in the real continuous space. Therefore, it is necessary to fit the difference pyramid to find the exact position and scale of the extreme points. Here, we use Taylor expansion as the fitting function.

[0131] In addition, we also need to remove extreme points at the edge to improve the stability of key points. Since the principal curvature of points at the edge of an object in an image is generally higher, we can use the principal curvature to determine whether the point is at the edge of the object. The principal curvature at a pixel point can be calculated using a two-dimensional Hessian matrix.

[0132] 3. Determine the direction of the feature points. In order to achieve image rotation without deformation, it is necessary to assign values ​​to the feature point directions according to the local image structure of the detected feature points. We use the image gradient histogram method to find the stable direction of the local structure of the feature points. After accurately locating the key points, we also find the scale value of the feature points. Based on this scale value, we get the Gaussian image closest to this scale value:

[0133]

[0134] Use finite differences to calculate the amplitude m(x, y) and argument θ(x, y) of the pixel points in the area centered on the feature point. The formula is as follows:

[0135]

[0136] 4. Feature point description vector: Use a set of vectors to describe the feature point. This description includes not only the feature point, but also the pixels around the feature point that contribute to it. It is used as the basis for target matching (so the descriptor should have a high uniqueness to ensure the matching rate). It can also make the feature point have more invariant characteristics, such as lighting changes, 3D viewpoint changes, etc.

[0137] (1) In order to ensure that the feature vector is rotationally invariant, it is necessary to rotate the feature point by an angle θ in the vicinity, that is, to rotate in the direction of the feature point.

[0138] (2) Divide the rotated area into 4*4 sub-areas, calculate the gradient histograms of 8 directions in the sub-areas, draw the cumulative value of the gradient direction in each direction, and form a seed point. Different from finding the main direction, in this case, the gradient direction histogram of each sub-area divides 0° to 360° into 8 direction intervals, each interval is 45°. That is, each seed point has the gradient intensity information of 8 direction intervals. Since there are 4*4 sub-areas, there are finally 4*4*8=128 dimensions of vector data.

[0139] Specifically, the steps of extracting the global feature vector through the feature extraction model are as follows:

[0140] (1) Data preprocessing: obtaining the inkjet coding image in the first feature area.

[0141] (2) Construct a triplet of coding images (coding image A1, coding image A2, coding image B), where coding image A2 is obtained from coding image A1 through data enhancement.

[0142] (3) Use the residual network Resnet to extract the features of the three inkjet images respectively.

[0143] (4) Using L2 regularization, the features of the three coding images are mapped into 128-dimensional feature vectors respectively.

[0144] (5) Calculate the ternary loss function and update the gradient. Take the coding images A1 and A2 as positive sample pairs, and the coding images A1 and B as negative sample pairs, and calculate the ternary loss function.

[0145] Specifically, the design of the feature matching model is as follows: Figure 3 As shown in the figure, it consists of three fully connected layers, each of which uses ReLU as the activation function, and the third fully connected layer finally outputs the result by Softmax.

[0146] The feature matching model is trained by minimizing the binary cross entropy loss L:

[0147]

[0148] Among them, y i represents the combined eigenvector pair x i If the combined feature vector pair is from the coding image A1 and the coding image A2, then y i is represented as 1; if the combined feature vector pair comes from the coding image A1 and the coding image B, then y i Indicated as 0. i and 1-p i Represents the binary value calculated from the Softmax following the fully connected layer F3.

[0149] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0150] Embodiment 2

[0151] In one embodiment, Figure 4 As shown, a product identification device is provided, and the device includes:

[0152] An acquisition module, the acquisition module is used to acquire an appearance image of the product to be identified;

[0153] An identification module, the identification module is used to identify a first feature area of ​​the appearance image of the product to be identified and first data information of the first feature area through a feature recognition model;

[0154] An extraction module, the extraction module is used to perform local feature extraction and global feature extraction on the first feature area through a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0155] A splicing module, configured to splice the first local feature vector and the first global feature vector into a first combined feature vector;

[0156] A search module, the search module is used to search the second combined feature vector of the product to be identified in the first database based on the first data information;

[0157] A comparison module, wherein the comparison module is used to input the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and output a comparison result and a comparison confidence; in response to a comparison result of failure, return a prompt of product identification failure; in response to a comparison result of success, return the first data information, and store the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence in the first database.

[0158] In one embodiment, before the acquisition module is used to acquire the appearance image of the product to be identified, the acquisition module is also used to acquire the appearance image of the product that has not yet been shipped;

[0159] The recognition module is further used to recognize a second feature area of ​​the appearance image of the unshipped product and second data information of the second feature area through the feature recognition model, wherein the unshipped product refers to a product to be identified that is in an unshipped state, and the second feature area is the same as the first feature area;

[0160] The extraction module is also used to perform local feature extraction and global feature extraction on the second feature area through the feature extraction model to obtain a second local feature vector and a second global feature vector;

[0161] The splicing module is further used to splice the second local feature vector and the second global feature vector into a second combined feature vector;

[0162] The device also includes a building module and a storage module:

[0163] The construction module is used to construct an association relationship between the second data information and the second combined feature vector;

[0164] The storage module is used to store the second data information and the second combined feature vector in the first database based on the association relationship.

[0165] In one embodiment, the acquisition module and the identification module are further used to:

[0166] According to a preset area size, a first feature area of ​​the appearance image of the product to be identified is identified by a text detection algorithm;

[0167] The first data information is identified by a target detection algorithm for a first feature area of ​​the appearance image of the product to be identified, wherein the first data information includes the production year, production batch and production number of the product to be identified.

[0168] In one embodiment, the extraction module is further used to:

[0169] Establishing a Gaussian pyramid based on the first feature area, generating a differential pyramid based on the Gaussian pyramid, and determining coordinates of potential feature points in the differential pyramid;

[0170] Based on the coordinates of the potential feature points, the feature points are fitted by Taylor expansion, and the coordinates of the actual feature points are obtained according to the fitting results, and the edge feature points in the actual feature points are removed by a corner point detection algorithm to obtain n final feature points, where n is a positive integer;

[0171] Acquire the scale value of the final feature point, and perform finite difference on the final feature point based on the scale value to obtain the amplitude and argument of the pixel points in the area centered on the final feature point;

[0172] The region centered on the final feature point is divided into 4*4 sub-regions, and the gradient histograms in 8 directions in each sub-region are calculated to generate a first local feature vector with n*128 dimensions.

[0173] In one embodiment, the extraction module is further used to:

[0174] Based on the first feature area, obtaining a first coding image;

[0175] Extracting features of the first coding image through a residual network;

[0176] The features of the first coding image are mapped into a first global feature vector of 128 dimensions through regularization.

[0177] In one embodiment, the splicing module is further used for:

[0178] Concatenate the first local feature vector of n*128 dimensions with the first global feature vector of 128 dimensions to obtain a first combined feature vector of (n+1)*128 dimensions;

[0179] The second local feature vector of n*128 dimensions is concatenated with the second global feature vector of 128 dimensions to obtain a second combined feature vector of (n+1)*128 dimensions;

[0180] The first combined feature vector of (n+1)*128 dimensions is concatenated with the second combined feature vector of (n+1)*128 dimensions to obtain a combined feature vector pair of (n+1)*256 dimensions;

[0181] The comparison module is also used for:

[0182] The combined feature vector of the (n+1)*256 dimensions is compared with the input feature matching model, and the comparison result and comparison confidence are output.

[0183] In one embodiment, the feature matching model is obtained by training the following modules:

[0184] A data enhancement module, the data enhancement module is used to perform data enhancement on the first coding image to obtain a second coding image;

[0185] The acquisition module is further used to acquire a third coding picture in the second database, wherein the third coding picture is any picture in the second database that is unrelated to the first coding picture;

[0186] A construction module, the construction module is used to construct a coding picture triplet based on the first coding picture, the second coding picture and the third coding picture;

[0187] The extraction module is also used to extract the features of the second coding image and the third coding image respectively through the residual network;

[0188] A mapping module, the mapping module is used to map the features of the second coding image and the third coding image into global feature vectors of 128 dimensions respectively through regularization;

[0189] A generating module, the generating module is used to generate local feature vectors of the second coding image and the third coding image respectively;

[0190] The splicing module is also used to splice the local feature vectors and the global feature vectors of the second coding image and the third coding image into a combined feature vector of (n+1)*128 dimensions; splice the combined feature vector of the first coding image with the combined feature vector of the second coding image to obtain a first combined feature vector pair; splice the combined feature vector of the first coding image with the combined feature vector of the third coding image to obtain a second combined feature vector pair;

[0191] The comparison module is further used to compare the first combined feature vector with the input feature matching model to obtain the similarity between the first coding image and the second coding image; compare the second combined feature vector with the input feature matching model to obtain the similarity between the first coding image and the third coding image;

[0192] An adjustment module is used to adjust the parameters of the feature matching model until the similarity between the first coding image and the second coding image is close to 1, and the similarity between the first coding image and the third coding image is close to 0.

[0193] For the specific definition of the product identification device, please refer to the definition of the product identification method above, which will not be repeated here. Each module in the above-mentioned product identification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0194] Embodiment 3

[0195] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0196] Acquire an appearance image of a product to be identified, and identify a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model;

[0197] Performing local feature extraction and global feature extraction on the first feature region by using a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0198] concatenating the first local feature vector and the first global feature vector into a first combined feature vector;

[0199] Based on the first data information, searching for a second combined feature vector of the product to be identified in the first database;

[0200] Inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence;

[0201] In response to the comparison result being a failure, returning a prompt indicating that the product identification has failed;

[0202] In response to a successful comparison result, the first data information is returned, and the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence are stored in the first database.

[0203] When the program instructions are read and executed by the one or more processors, the program instructions may also perform operations corresponding to the steps in the above method embodiments, and the above description may be referred to, which will not be repeated here. Figure 5, which exemplarily shows the architecture of a computer device, which may specifically include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520 may be communicatively connected via a communication bus 530.

[0204] Among them, the processor 510 can be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0205] The memory 520 can be implemented in the form of a read-only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the computer device 500, and a basic input and output system (BIOS) 522 for controlling the low-level operation of the computer device 500. In addition, a web browser 523, a data storage manager 524, and an icon font processing system 525, etc. can also be stored. The above-mentioned icon font processing system 525 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510.

[0206] The input / output interface 513 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0207] The network interface 514 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0208] The bus 530 comprises a pathway for transmitting information between the various components of the device (eg, the processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520).

[0209] In addition, the computer device 500 can also obtain information on specific collection conditions from the virtual resource object collection condition information database 541 for use in condition judgment, etc.

[0210] It should be noted that, although the above-mentioned computer device 500 only shows a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, a memory 520, a bus 530, etc., in the specific implementation process, the computer device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above-mentioned device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0211] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a cloud server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0212] Embodiment 4

[0213] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0214] Acquire an appearance image of a product to be identified, and identify a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model;

[0215] Performing local feature extraction and global feature extraction on the first feature region by using a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0216] concatenating the first local feature vector and the first global feature vector into a first combined feature vector;

[0217] Based on the first data information, searching for a second combined feature vector of the product to be identified in the first database;

[0218] Inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence;

[0219] In response to the comparison result being a failure, returning a prompt indicating that the product identification has failed;

[0220] In response to a successful comparison result, the first data information is returned, and the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence are stored in the first database.

[0221] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0222] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0223] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

[0224] Embodiment 5

[0225] In one embodiment, a computer program product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0226] Acquire an appearance image of a product to be identified, and identify a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model;

[0227] Performing local feature extraction and global feature extraction on the first feature region by using a feature extraction model to obtain a first local feature vector and a first global feature vector;

[0228] concatenating the first local feature vector and the first global feature vector into a first combined feature vector;

[0229] Based on the first data information, searching for a second combined feature vector of the product to be identified in the first database;

[0230] Inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence;

[0231] In response to the comparison result being a failure, returning a prompt indicating that the product identification has failed;

[0232] In response to a successful comparison result, the first data information is returned, and the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence are stored in the first database.

[0233] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a computer program product. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0234] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0235] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A product identification method, characterized in that: The method comprises: Acquire an appearance image of a product to be identified, and identify a first feature region of the appearance image of the product to be identified and first data information of the first feature region through a feature recognition model; Performing local feature extraction and global feature extraction on the first feature region by using a feature extraction model to obtain a first local feature vector and a first global feature vector; concatenating the first local feature vector and the first global feature vector into a first combined feature vector; Based on the first data information, searching for a second combined feature vector of the product to be identified in the first database; Inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence; In response to the comparison result being a failure, returning a prompt indicating that the product identification has failed; In response to a successful comparison result, the first data information is returned, and the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence are stored in the first database.

2. The method according to claim 1, characterized in that: Before obtaining the appearance image of the product to be identified, the method further includes: Acquire an appearance image of a product that has not been shipped out of the factory, and identify a second feature region of the appearance image of the product that has not been shipped out of the factory and second data information of the second feature region through the feature recognition model, wherein the product that has not been shipped out of the factory refers to a product to be identified that has not been shipped out of the factory, and the second feature region is the same as the first feature region; Performing local feature extraction and global feature extraction on the second feature region by using the feature extraction model to obtain a second local feature vector and a second global feature vector; concatenating the second local feature vector and the second global feature vector into a second combined feature vector; An association relationship between the second data information and the second combined feature vector is constructed, and based on the association relationship, the second data information and the second combined feature vector are stored in the first database.

3. The method according to claim 1, characterized in that: The step of acquiring the appearance image of the product to be identified, and identifying the first feature region of the appearance image of the product to be identified and the first data information of the first feature region by using the feature recognition model includes: According to a preset area size, a first feature area of ​​the appearance image of the product to be identified is identified by a text detection algorithm; The first data information is identified by a target detection algorithm for a first feature area of ​​the appearance image of the product to be identified, wherein the first data information includes the production year, production batch and production number of the product to be identified.

4. The method according to claim 1, characterized in that: The extracting local features of the first feature region by using a feature extraction model to obtain a first local feature vector includes: Establishing a Gaussian pyramid based on the first feature area, generating a differential pyramid based on the Gaussian pyramid, and determining coordinates of potential feature points in the differential pyramid; Based on the coordinates of the potential feature points, the feature points are fitted by Taylor expansion, and the coordinates of the actual feature points are obtained according to the fitting results, and the edge feature points in the actual feature points are removed by a corner point detection algorithm to obtain n final feature points, where n is a positive integer; Acquire the scale value of the final feature point, and perform finite difference on the final feature point based on the scale value to obtain the amplitude and argument of the pixel points in the area centered on the final feature point; The region centered on the final feature point is divided into 4*4 sub-regions, and the gradient histograms in 8 directions in each sub-region are calculated to generate a first local feature vector with n*128 dimensions.

5. The method according to claim 4, characterized in that: Performing global feature extraction on the first feature region through a feature extraction model to obtain a first global feature vector includes: Based on the first feature area, obtaining a first coding image; Extracting features of the first coding image through a residual network; The features of the first coding image are mapped into a first global feature vector of 128 dimensions through regularization.

6. The method according to claim 5, characterized in that: The step of concatenating the first local feature vector and the first global feature vector into a first combined feature vector includes: Concatenate the first local feature vector of n*128 dimensions with the first global feature vector of 128 dimensions to obtain a first combined feature vector of (n+1)*128 dimensions; The step of concatenating the second local feature vector and the second global feature vector into a second combined feature vector comprises: The second local feature vector of n*128 dimensions is concatenated with the second global feature vector of 128 dimensions to obtain a second combined feature vector of (n+1)*128 dimensions; The step of inputting the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and outputting a comparison result and a comparison confidence level includes: The first combined feature vector of (n+1)*128 dimensions is concatenated with the second combined feature vector of (n+1)*128 dimensions to obtain a combined feature vector pair of (n+1)*256 dimensions; The combined feature vector of the (n+1)*256 dimensions is compared with the input feature matching model, and the comparison result and comparison confidence are output.

7. The method according to claim 5, characterized in that: The feature matching model is trained in the following way: Performing data enhancement on the first coding image to obtain a second coding image; Obtain a third coding picture in the second database, and construct a coding picture triplet based on the first coding picture, the second coding picture and the third coding picture, wherein the third coding picture is any picture in the second database that is unrelated to the first coding picture; Extracting features of the second coding image and the third coding image respectively through the residual network; Mapping the features of the second coding image and the third coding image into global feature vectors of 128 dimensions respectively through regularization; Generate local feature vectors of the second coding image and the third coding image respectively, and concatenate the local feature vectors of the second coding image and the third coding image with the global feature vector into a combined feature vector of (n+1)*128 dimensions; The combined feature vector of the first coding image is concatenated with the combined feature vector of the second coding image to obtain a first combined feature vector pair, and the first combined feature vector pair is input into a feature matching model for comparison to obtain the similarity between the first coding image and the second coding image; The combined feature vector of the first coding image is concatenated with the combined feature vector of the third coding image to obtain a second combined feature vector pair, and the second combined feature vector pair is input into a feature matching model for comparison to obtain the similarity between the first coding image and the third coding image; The parameters of the feature matching model are adjusted until the similarity between the first coding image and the second coding image is close to 1, and the similarity between the first coding image and the third coding image is close to 0.

8. A product identification device, characterized in that: The device comprises: An acquisition module, the acquisition module is used to acquire an appearance image of the product to be identified; An identification module, the identification module is used to identify a first feature area of ​​the appearance image of the product to be identified and first data information of the first feature area through a feature recognition model; An extraction module, the extraction module is used to perform local feature extraction and global feature extraction on the first feature area through a feature extraction model to obtain a first local feature vector and a first global feature vector; A splicing module, configured to splice the first local feature vector and the first global feature vector into a first combined feature vector; A search module, the search module is used to search the second combined feature vector of the product to be identified in the first database based on the first data information; A comparison module, wherein the comparison module is used to input the first combined feature vector and the second combined feature vector into a feature matching model for comparison, and output a comparison result and a comparison confidence; in response to a comparison result of failure, return a prompt of product identification failure; in response to a comparison result of success, return the first data information, and store the appearance image of the product to be identified, the first data information, the comparison result and the comparison confidence in the first database.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Code fingerprint anti-counterfeiting method

    CN110288359A

  • Image retrieval method based on multi-feature fusion

    CN114140657A

  • Code spraying pattern detection method and device, electronic equipment and storage medium

    CN116469090A

  • Object recognition apparatus and method based on environment matching

    US20230015295A1

Cited By

  • Food authenticity identification method and system based on spectral imaging

    CN120028284A