Product tracing information query method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of quality management, and particularly relates to a product traceability information query method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized image, including a target product image, a target text image and a target traceability code image of a to-be-recognized product; inputting the to-be-recognized image into a trained image big data analysis model, and obtaining a target product image analysis result, a target text image analysis result and a target traceability code image analysis result based on a product image analysis model, a text image analysis model and a traceability code image analysis model in the image big data analysis model; based on the three analysis results, first product tracing information, second product tracing information and third product tracing information are correspondingly inquired from a product tracing information database; based on the three pieces of product tracing information, the target product tracing information of the to-be-identified commodity is determined, and the accuracy of querying the product tracing information is improved.

Description

Technical Field

[0001] The present application belongs to the field of quality management technology, and in particular, relates to a product traceability information query method, device, equipment and storage medium. Background Art

[0003] At present, most product quality and safety traceability platforms use product traceability codes to determine product identity and traceability. Users usually query product traceability information by scanning the traceability QR code on the product packaging. However, this method has many problems. For example, when the product traceability code label carrier is incomplete or missing, or when the traceability QR code on the product packaging is maliciously replaced, it will be impossible to accurately obtain product traceability information.

[0004] Therefore, how to accurately query product traceability information has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a product traceability information query method, device, equipment and storage medium, aiming to accurately query product traceability information.

[0006] In a first aspect, an embodiment of the present application provides a product traceability information query method, the method comprising: obtaining an image to be identified, the image to be identified comprising a target product image, a target text image, and a target traceability code image of the product to be identified; inputting the image to be identified into a trained image big data analysis model, and obtaining a target product image analysis result, a target text image analysis result, and a target traceability code image analysis result based on the product image analysis model, the text image analysis model, and the traceability code image analysis model in the image big data analysis model; obtaining first product traceability information, second product traceability information, and third product traceability information from a product traceability information database based on the target product image analysis result, the target text image analysis result, and the target traceability code image analysis result; and determining the target product traceability information of the product to be identified based on the first product traceability information, the second product traceability information, and the third product traceability information.

[0007] In a possible implementation, determining the target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information, and the third product traceability information includes: judging whether the first product traceability information, the second product traceability information, and the third product traceability information are the same; when at least two of them are the same, determining the same product traceability information as the target product traceability information of the product to be identified.

[0008] In a possible implementation, determining the target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information, and the third product traceability information includes: judging whether the third product traceability information satisfies a first preset condition, and if so, determining the third product traceability information as the target product traceability information; if not, judging whether the second product traceability information satisfies a second preset condition, and if so, determining the second product traceability information as the target product traceability information; if not, determining the first product traceability information as the target product traceability information.

[0009] In a possible implementation, the training process of the image big data analysis model includes: acquiring an image dataset, the image dataset including a product image dataset, a text image dataset and a traceability code image dataset; based on the product image dataset, constructing and training a deep convolutional neural network model to obtain the product image analysis model with the product image as input and the product image analysis result as output, the product image analysis result including a feature vector of the product image; based on the text image dataset, constructing and training a text recognition model to obtain a text image analysis model with the text image as input and the text image analysis result as output, the text image analysis result including location information of the product text; based on the traceability code image dataset, constructing and training an object detection model to obtain a traceability code image analysis model with the traceability code image as input and the traceability code image analysis result as output, the traceability code image analysis result including location information of the product traceability code; based on the product image analysis model, the text image analysis model and the traceability code image analysis model, constructing the image big data analysis model.

[0010] In a possible implementation, the target product image analysis result includes a target product image feature vector of the product to be identified, the target text image analysis result includes target product text position information of the product to be identified, and the target traceability code image analysis result includes target product traceability code position information of the product to be identified; and the querying and obtaining the first product traceability information, the second product traceability information, and the third product traceability information from a product traceability information database based on the target product image analysis result, the target text image analysis result, and the target traceability code image analysis result, respectively, includes: determining target category information of the product to be identified based on the product image data set and the target product image feature vector, querying and obtaining the first product traceability information from the product traceability information database based on the target category information; decoding the target product text position information to obtain the target text information of the product to be identified, and querying and obtaining the second product traceability information from the product traceability information database based on the target text information; decoding the target product traceability code position information using a target decoder to obtain the target traceability code information of the product to be identified, and querying and obtaining the third product traceability information from the product traceability information database based on the target traceability code information.

[0011] In a possible implementation, determining the target category information of the product to be identified based on the product image dataset and the target product image feature vector includes: vectorizing each product image in the product image dataset to obtain the feature vector of each product image and storing it in a vector database to obtain a product image database; calculating the similarity between the feature vector of the target product image and the feature vector of each product image in the product image database to obtain a similarity ranking; determining a similar image corresponding to the target product image based on the similarity ranking; and determining the target category information of the product to be identified based on the category information corresponding to the similar image.

[0012] In a possible implementation, the target decoder is a dual decoder, and the target decoder is used to decode the target product traceability code position information to obtain the target traceability code information of the product to be identified, including: based on the target product traceability code position information, intercepting a target traceability code image area from the image to be identified according to a preset ratio, and the preset ratio is greater than 1; based on the dual decoder, the target traceability code image area is decoded to obtain the target traceability code information of the product to be identified.

[0013] In a second aspect, an embodiment of the present application provides a product traceability information query device, the device comprising: an image acquisition module, used to acquire an image to be identified, the image to be identified comprising a target product image, a target text image and a target traceability code image of the product to be identified; a result analysis module, used to input the image to be identified into a trained image big data analysis model, and obtain a target product image analysis result, a target text image analysis result and a target traceability code image analysis result based on the product image analysis model, the text image analysis model and the traceability code image analysis model in the image big data analysis model; an information query module, used to query and obtain first product traceability information, second product traceability information and third product traceability information from a product traceability information database based on the target product image analysis result, the target text image analysis result and the target traceability code image analysis result; an information determination module, used to determine the target product traceability information of the product to be identified based on the first product traceability information, the second product traceability information and the third product traceability information.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the method described in the first aspect or any one of the implementation methods thereof.

[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects: an image to be identified including a target product image, a target text image and a target traceability code image of the product to be identified is obtained; based on the target product image, the target text image and the target traceability code image, a product image analysis model, a text image analysis model and a traceability code image analysis model in the image big data analysis model are used to obtain first product traceability information, second product traceability information and third product traceability information respectively corresponding to the target product image, the target text image and the target traceability code image; and target product traceability information of the commodity to be identified is determined based on the obtained three product traceability information. Compared with querying product traceability information only through the traceability code on the product packaging, the step of querying product traceability information through the product image and text image of the product to be identified is added, which effectively avoids the situation where the product traceability information cannot be queried when the product traceability code label carrier is incomplete or missing, and ensures that the product traceability information can be queried under any circumstances; in addition, the target product traceability information of the image to be identified is determined by the first product traceability information, the second product traceability information and the third product traceability information obtained through the target product image, the target text image and the target traceability code image respectively, which solves the problem of querying wrong product traceability information when the traceability code on the product packaging is maliciously replaced, and ensures the accuracy of the queried product traceability information.

[0018] It can be understood that the product traceability information query device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application have the same beneficial effects as the above-mentioned product traceability information query method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of the architecture of a product traceability information query system provided in one embodiment of the present application;

[0021] Figure 2 A flowchart of a product traceability information query method provided in one embodiment of the present application;

[0022] Figure 3 A schematic diagram of a product traceability information query method provided in one embodiment of the present application;

[0023] Figure 4 A schematic diagram of an image to be recognized provided in one embodiment of the present application;

[0024] Figure 5 A schematic diagram of an image recognition result provided by an embodiment of the present application;

[0025] Figure 6 A flowchart of a method for training an image big data analysis model provided in one embodiment of the present application;

[0026] Figure 7 A structural block diagram of a product traceability information query device provided in one embodiment of the present application;

[0027] Figure 8 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0029] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0030] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0031] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0032] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0033] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0034] For ease of understanding, the technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0035] Figure 1 This is a schematic diagram of the architecture of a product traceability information query system provided in one embodiment of the present application. Figure 1 As shown, the product traceability information query system includes an image uploading module, an image display module, an image big data analysis module, a recognition result display module, and a traceability information display module.

[0036] Specifically, the image upload module is used by the user to upload the image to be identified of the product to be identified locally or from the network; the image display module is used to display the image to be identified uploaded by the user; the image big data analysis module is used to perform intelligent recognition and analysis on the image to be identified uploaded by the user. The module applies the product traceability information query method provided by the present application, firstly through the image to be identified uploaded by the user and the pre-trained image big data training model, intelligently identifies the product information, text information and traceability code information and other product-related information of the product to be identified, and then automatically associates the product traceability information database based on the identified product-related information to obtain the product traceability information; the recognition result display module is used to display the image recognition result obtained after the recognition and analysis of the image to be identified, specifically including the recognition results of the product, text and traceability code, which are marked with rectangular frames respectively; the traceability information display module is used to display the product traceability information of the product to be identified.

[0037] Understandably, Figure 1 The schematic diagram of the architecture shown is only an example of the product traceability information query system provided by the present application. In other embodiments of the present application, the product traceability information query system may include more or fewer components than shown in the diagram, or combine certain components, or split certain components, or arrange the components differently. The components shown in the diagram may be implemented in hardware, software, or a combination of software and hardware, and the present application does not limit this.

[0038] Figure 2A flowchart of a product traceability information query method provided in an embodiment of the present application is shown in FIG. Figure 3 A schematic diagram of a product traceability information query method provided in an embodiment of the present application is shown as follows: Figure 2 and Figure 3 For the sake of convenience, only the parts related to this embodiment are shown. The method provided in this embodiment includes the following steps:

[0039] S210, obtaining an image to be identified, where the image to be identified includes a target product image, a target text image, and a target traceability code image of the product to be identified.

[0040] Specifically, the target product image of the product to be identified is an image containing the product to be identified, and the target product image usually contains a part that can show the characteristics of the product to be identified, so as to accurately identify the category of the product to be identified; the product appearance packaging generally has text information related to the product printed on it, such as the product name, production date, production batch, etc., and the target text image of the product to be identified is an image containing the text information on the product to be identified; the traceability code of the product quality and safety traceability platform is generally affixed to the product packaging in the form of a label, and the target traceability code image of the product to be identified is an image containing the traceability code of the product to be identified, wherein the traceability code of the product to be identified is generally a one-dimensional code or a two-dimensional code, etc.

[0041] Optionally, the image to be identified may be a single photo or multiple photos, or one or more frames of images in a video, which is not limited in this application.

[0042] In a specific implementation, a camera or a video camera is used to photograph the product to be identified to obtain an image to be identified. An example of an image to be identified is as follows: Figure 4 shown.

[0043] S220, input the image to be recognized into the trained image big data analysis model, and obtain the target product image analysis result, target text image analysis result and target traceability code image analysis result based on the product image analysis model, text image analysis model and traceability code image analysis model in the image big data analysis model respectively.

[0044] In a specific implementation, based on the target product image in the image to be identified, the target product image analysis result is obtained through the product image analysis model in the image big data analysis model; based on the target text image in the image to be identified, the target text image analysis result is obtained through the text image analysis model in the image big data analysis model; based on the target traceability code image in the image to be identified, the target traceability code image analysis result is obtained through the traceability code image analysis model in the image big data analysis model. Specifically, the image big data analysis model can adopt any structural form. The first inventive point of this application is to integrate the various analysis results to obtain more accurate traceability results more conveniently and quickly.

[0045] As an example, the target product image analysis result, the target text image analysis result and the target traceability code image analysis result respectively include the product to be identified, the text information and the position information of the traceability code in the image to be identified. Figure 5 A schematic diagram of an image recognition result provided by an embodiment of the present application, such as Figure 5 As shown, box 1 is the target product image analysis result, box 2 is the target text image analysis result, and box 3 is the target traceability code image analysis result.

[0046] S230, based on the target product image analysis result, the target text image analysis result and the target traceability code image analysis result, respectively, query the product traceability information database to obtain first product traceability information, second product traceability information and third product traceability information.

[0047] Specifically, the product traceability information database refers to a system used to store and manage traceability record data in various links from production to sales, including key information such as production date, production batch, source of raw materials, quality inspection report, etc.; the product traceability information queried from the product traceability information database is the above-mentioned traceability record data corresponding to the product to be identified recorded in the product traceability information database.

[0048] In a specific implementation, information such as the target product category of the product to be identified is determined based on the target product image analysis result, and first product traceability information is obtained by querying from a product traceability information database based on the target product category and other information; text information such as the target product name of the product to be identified is determined based on the target text image analysis result, and second product traceability information is obtained by querying from a product traceability information database based on the target product name and other text information; target traceability code information of the product to be identified is determined based on the target traceability code image analysis result, and third product traceability information is obtained by querying from a product traceability information database based on the target traceability code information, wherein the target traceability code information includes image information or codeword information of the traceability code of the product to be identified.

[0049] S240: Determine target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information, and the third product traceability information.

[0050] In a possible implementation, it is determined whether the first product traceability information, the second product traceability information, and the third product traceability information are the same; when at least two of them are the same, the same product traceability information is determined as the target product traceability information of the product to be identified.

[0051] Specifically, to determine whether the first product traceability information is the same as the second product traceability information is to determine whether all data in the first product traceability information is the same as all data in the second product traceability information. For example, to determine whether the production date in the first product traceability information is the same as the production date in the second product traceability information, and whether the production batch in the first product traceability information is the same as the production batch in the second product traceability information.

[0052] As an example, when the first product traceability information, the second product traceability information and the third product traceability information are all the same, any one of the first product traceability information, the second product traceability information and the third product traceability information is determined as the target product traceability information.

[0053] As another example, when the first product traceability information is the same as the second product traceability information but different from the third product traceability information, any one of the first product traceability information and the second product traceability information is determined as the target product traceability information.

[0054] As another example, when the first product traceability information, the second product traceability information, and the third product traceability information are all different, querying the target product traceability information fails.

[0055] This implementation method ensures the accuracy of querying the traceability information of the target product, and avoids the situation where errors in any of the first product traceability information, the second product traceability information, and the third product traceability information affect the final target product traceability information.

[0056] In another possible implementation, it is determined whether the third product traceability information meets the first preset condition. If so, the third product traceability information is determined as the target product traceability information; if not, it is determined whether the second product traceability information meets the second preset condition. If so, the second product traceability information is determined as the target product traceability information; if not, the first product traceability information is determined as the target product traceability information.

[0057] Specifically, the first preset condition is that the third product traceability information is not empty and the traceability code corresponding to the third product traceability information is the traceability code in the product quality and safety traceability platform; the second preset condition is that the second product traceability information is not empty and the text information corresponding to the second product traceability information is correct.

[0058] In the specific implementation, it is first determined whether the third product traceability information is not empty and whether the traceability code corresponding to the third product traceability information is the traceability code in the product quality and safety traceability platform. If so, the third product traceability information is determined as the target product traceability information; if not, it is determined whether the second product traceability information is not empty and whether the text information corresponding to the second product traceability information, such as product name, production batch, production date, etc., is correct. If so, the second product traceability information is determined as the target product traceability information; if not, the first product traceability information is determined as the target product traceability information.

[0059] This implementation solves the problem that product traceability information cannot be queried or cannot be correctly queried through the traceability code, ensuring that product traceability information can be queried under any circumstances.

[0060] The technical solution provided by the present application obtains an image to be identified including a target product image, a target text image and a target traceability code image of the product to be identified, and based on the target product image, the target text image and the target traceability code image, adopts a product image analysis model, a text image analysis model and a traceability code image analysis model in an image big data analysis model to obtain first product traceability information, second product traceability information and third product traceability information corresponding to the target product image, the target text image and the target traceability code image, respectively, and determines the target product traceability information of the commodity to be identified based on the obtained three product traceability information. Compared with querying product traceability information only through the traceability code on the product packaging, the step of querying product traceability information through the product image and text image of the product to be identified is added, which effectively avoids the situation where the product traceability information cannot be queried when the product traceability code label carrier is incomplete or missing, and ensures that the product traceability information can be queried under any circumstances; in addition, the target product traceability information of the image to be identified is determined by obtaining the first product traceability information, the second product traceability information and the third product traceability information through the target product image, the target text image and the target traceability code image respectively, which solves the problem of querying wrong product traceability information when the traceability code on the product packaging is maliciously replaced, and ensures the accuracy of the queried product traceability information.

[0061] Figure 6 A flowchart of a method for training an image big data analysis model provided in one embodiment of the present application. Figure 6As shown, based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the training process of the image big data analysis model includes:

[0062] S610, acquiring an image dataset, where the image dataset includes a product image dataset, a text image dataset, and a traceability code image dataset.

[0063] In the specific implementation, product images are obtained from the product quality and safety traceability platform or collected through a web crawler to obtain a product image data set and aggregate them to form a product image database; text images corresponding to the products are obtained from the product quality and safety traceability platform or text images of the products are collected through a web crawler to obtain a text image data set and aggregate them to form a text image database; traceability code images are obtained from the product quality and safety traceability platform or traceability code images are collected through a web crawler to obtain a traceability code image data set and aggregate them to form a traceability code image database.

[0064] S620, based on the product image dataset, construct and train a deep convolutional neural network model to obtain a product image analysis model with the product image as input and the product image analysis result as output, where the product image analysis result includes a feature vector of the product image.

[0065] Specifically, the training process of the product image analysis model mainly converts the product images in the product image database into feature vectors, and then writes them into the vector database for storage.

[0066] Preferably, the Resnet50 deep convolutional neural network model is used to vectorize the features of product images, and the Milvus vector database is used for storage and query of feature vectors.

[0067] In the specific implementation, the product images in the product image database are first normalized, scaled, cropped and other preprocessing operations are performed to meet the input requirements of the ResNet50 model; the pre-trained ResNet50 model is loaded using the PyTorch framework, and the pre-processed product images are input into the ResNet50 model for feature vector extraction, and the features before the last fully connected layer of the ResNet50 model are used as the feature vectors of each product image; a Milvus collection is created, and the name, dimension (consistent with the feature vector dimension) and index type of the collection are defined. For each product image in the product image database, the feature vector is extracted and stored in the Milvus collection.

[0068] S630, constructing and training a text recognition model based on the text image data set to obtain a text image analysis model that takes the text image as input and outputs a text image analysis result, wherein the text image analysis result includes location information of the product text.

[0069] Specifically, the SVTR text recognition model is adopted, and the text image analysis model is defined according to the SVTR model architecture, including a vision transformer (ViT) style encoder, position encoding, attention mechanism, feedforward network, etc. The connectionist temporal classification (CTC) is used as the loss function, and the training parameters such as learning rate, optimizer (such as Adam), batch size, and number of iterations are defined. The model is initialized using pre-trained weights.

[0070] In the specific implementation, the labeling tool LabelImg is used to label each text image in the text image database and generate the corresponding text label. The text image size is adjusted to a unified size of (32, 128, 3), the text label is converted to a unified UTF-8 encoding format, and data enhancement techniques such as rotation, scaling, shearing, and color jittering are applied to improve the generalization ability of the model. The text image dataset is divided into a training set, a validation set, and a test set with a split ratio of 8:1:1. The training set is used to train the text recognition model, the validation set is used to select and adjust the parameters and structure of the text recognition model, and the test set is used to evaluate the performance and accuracy of the text recognition model, so that the text image analysis model obtained after training has appropriate model parameters to achieve the best performance.

[0071] S640, based on the traceability code image data set, construct and train a target detection model to obtain a traceability code image analysis model that takes the traceability code image as input and outputs the traceability code image analysis result, wherein the traceability code image analysis result includes the location information of the product traceability code.

[0072] Preferably, the YOLOv8n model is selected as the traceability code image analysis model.

[0073] In the specific implementation, the labeling tool LabelImg is used to label each traceability code image in the traceability code image dataset, the area where the traceability code is located is framed, and the labeling file is saved in a text file (txt) format that can be recognized by the YOLOv8n model. The traceability code image and its labeling file are randomly divided into a training set and a validation set in a ratio of 8:2. The images folder and labels folder are set under the training set and the validation set to store the traceability code images and label files respectively. The images in the traceability code image dataset are input into the YOLOv8n model, the training parameters are set according to the configuration of the training device, and the performance of the trained model is evaluated using the validation set and map@0.5 to obtain the traceability code image analysis model.

[0074] S650 builds an image big data analysis model based on the product image analysis model, text image analysis model and traceability code image analysis model.

[0075] The trained product image analysis model, text image analysis model and traceability code image analysis model are constructed into an image big data analysis model.

[0076] The technical solution provided in this embodiment obtains image data sets from a product quality and safety traceability platform and through a web crawler, thereby improving the richness and diversity of the image data sets; the obtained image data sets are used to train a product image analysis model, a text image analysis model, and a traceability code image analysis model, respectively, and an image big data analysis model is constructed based on the trained product image analysis model, text image analysis model, and traceability code image analysis model, thereby improving the accuracy of the image big data analysis model.

[0077] On the basis of the above-mentioned embodiment, the present embodiment further illustrates and optimizes the technical solution. Specifically, in the present embodiment, the target product image analysis result includes the target product image feature vector of the product to be identified, the target text image analysis result includes the target product text position information of the product to be identified, and the target traceability code image analysis result includes the target product traceability code position information of the product to be identified; then, in step S230, based on the target product image analysis result, the target text image analysis result, and the target traceability code image analysis result, the first product traceability information, the second product traceability information, and the third product traceability information are obtained by correspondingly querying from the product traceability information database, including:

[0078] S2301, based on the product image data set and the target product image feature vector, determine the target category information of the product to be identified, and query the product traceability information database to obtain the first product traceability information based on the target category information.

[0079] In one possible implementation, the similarity between the feature vector of the target product image and the feature vectors of each product image in the product image database is calculated to obtain a similarity ranking; based on the similarity ranking, similar images corresponding to the target product image are determined; based on the category information corresponding to the similar images, target category information of the product to be identified is determined.

[0080] In a specific implementation, a similarity ranking is obtained by calculating the cosine similarity, dot product similarity or Euclidean distance between the feature vector of the target product image and the feature vectors of each product image in the product image database, and the product image corresponding to the feature vector of the product image with the highest similarity to the feature vector of the target product image in the similarity ranking is determined as a similar image of the target product image, and the product category, product name and other information corresponding to the similar image that can reflect the identity of the product to be identified is determined as the target category information of the product to be identified, and the first product traceability information is obtained by querying from the product traceability information database based on the target category information. The more data contained in the target category information of the product to be identified obtained based on the similar image, the more accurate the obtained first product traceability information.

[0081] S2302, decoding the target product text position information to obtain target text information of the product to be identified, and querying from a product traceability information database based on the target text information to obtain second product traceability information.

[0082] In a specific implementation, the weights of the trained text image analysis model are loaded, the target text image is preprocessed in the same manner as during training, the preprocessed target text image is input into the text image analysis model for inference, and the target product text position information is obtained. The target product text position information output by the model is decoded into a text format, and operations such as space removal and punctuation correction are performed to obtain the target text information of the product to be identified. Based on the target text information, the second product traceability information is obtained by querying from a product traceability information database, wherein the target text information includes information such as the product name, production date, and production batch number of the product to be identified.

[0083] S2303, using a target decoder to decode the target product traceability code position information to obtain the target traceability code information of the product to be identified, and querying from the product traceability information database based on the target traceability code information to obtain the third product traceability information.

[0084] In a possible implementation, based on the target product traceability code position information, a target traceability code image area is intercepted from the image to be identified according to a preset ratio, and the preset ratio is greater than 1; the target traceability code image area is decoded based on a dual decoder to obtain the target traceability code information of the product to be identified.

[0085] In the specific implementation, the target traceability code image of the product to be identified is input into the trained traceability code image analysis model to obtain the target product traceability code location information, that is, the coordinates of the target traceability code area, and the target traceability code area is intercepted by the open source computer vision library (Open Source Computer Vision Library, OpenCV) to obtain the target traceability code image area. In order to prevent errors during decoding, the target traceability code image area is intercepted according to a preset ratio, preferably, the preset ratio is 1.1; the intercepted target traceability code image area is then input into the target decoder for decoding. In this embodiment, QR code detector (QRCodeDetector) and WeChatQRCode dual decoder are used to decode the target traceability code image area, wherein QRCodeDetector is a class in the Opencv library, which is used to identify and locate QR codes from images, and supports various correction, tilt, and scaling QR code detection. WeChatQRCode is an open source project based on Opencv, which realizes the efficient recognition function of WeChat QR code. QRCodeDetector is used by default to decode the target traceability code image area, and WeChatQRCode is used for decoding if it fails.

[0086] In this implementation, the design of dual decoders can effectively improve the accuracy of traceability code recognition and parsing.

[0087] The technical solution provided by this embodiment first determines the target category information, target text information and target traceability code information of the product to be identified through the target product image feature vector, target product text position information and target product traceability code position information of the product to be identified, and then queries the product traceability information from the product traceability information database based on the target category information, target text information and target traceability code information, thereby improving the accuracy of querying product traceability information.

[0088] Figure 7 This is a structural block diagram of a product traceability information query device provided in an embodiment of the present application. For the sake of convenience, only the parts related to the embodiment of the present application are shown. Figure 7 The product traceability information query device 700 may include an image acquisition module 701 , a result analysis module 702 , an information query module 703 and an information determination module 704 .

[0089] The image acquisition module 701 is used to acquire the image to be identified, where the image to be identified includes a target product image, a target text image and a target traceability code image of the product to be identified.

[0090] The result analysis module 702 is used to input the image to be identified into the trained image big data analysis model, and obtain the target product image analysis result, target text image analysis result and target traceability code image analysis result based on the product image analysis model, text image analysis model and traceability code image analysis model in the image big data analysis model.

[0091] The information query module 703 is used to query the product traceability information database to obtain the first product traceability information, the second product traceability information and the third product traceability information based on the target product image analysis result, the target text image analysis result and the target traceability code image analysis result.

[0092] The information determination module 704 is used to determine the target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information and the third product traceability information.

[0093] A product traceability information query device provided in an embodiment of the present application has the same beneficial effects as the above-mentioned product traceability information query method.

[0094] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0095] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0096] Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 8 As shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8Only one is shown in the figure), a memory 81, and a computer program 82 stored in the memory 81 and executable on at least one processor 80, the processor 80 executes the computer program 82 to implement the above Figure 2 , Figure 3 , Figure 6 The steps in the method embodiment, or the implementation of the above Figure 7 Functions of each module / unit in the device embodiment.

[0097] The electronic device 8 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 8 may include but is not limited to a processor 80 and a memory 81. Those skilled in the art will appreciate that Figure 8 It is only an example of the electronic device 8 and does not constitute a limitation on the electronic device 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0098] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0099] In some embodiments, the memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. In other embodiments, the memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 8. Further, the memory 81 may also include both an internal storage unit of the electronic device 8 and an external storage device. The memory 81 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program, etc. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0100] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0102] A computer-readable storage medium provided in an embodiment of the present application has the same beneficial effects as the above-mentioned product traceability information query method.

[0103] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0104] A computer program product provided in an embodiment of the present application has the same beneficial effects as the above-mentioned product traceability information query method.

[0105] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0106] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0107] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A product traceability information query method, characterized in that: The method comprises: Acquire an image to be identified, wherein the image to be identified includes a target product image, a target text image, and a target traceability code image of the product to be identified; The image to be identified is input into a trained image big data analysis model, and based on the product image analysis model, the text image analysis model and the traceability code image analysis model in the image big data analysis model, a target product image analysis result, a target text image analysis result and a target traceability code image analysis result are obtained respectively; Based on the target product image analysis result, the target text image analysis result and the target traceability code image analysis result, respectively, query the product traceability information database to obtain first product traceability information, second product traceability information and third product traceability information; Based on the first product traceability information, the second product traceability information and the third product traceability information, target product traceability information of the commodity to be identified is determined.

2. The method according to claim 1, characterized in that The determining the target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information, and the third product traceability information includes: Determining whether the first product traceability information, the second product traceability information, and the third product traceability information are the same; When at least two of them are the same, the same product traceability information is determined as the target product traceability information of the product to be identified.

3. The method according to claim 1, characterized in that The determining the target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information, and the third product traceability information includes: Determine whether the third product traceability information meets the first preset condition, and if so, determine the third product traceability information as the target product traceability information; If not, determining whether the second product traceability information meets a second preset condition, and if so, determining the second product traceability information as the target product traceability information; If not, the first product traceability information is determined as the target product traceability information.

4. The method according to claim 1, characterized in that The training process of the image big data analysis model includes: Acquire an image dataset, wherein the image dataset includes a product image dataset, a text image dataset, and a traceability code image dataset; Based on the product image dataset, a deep convolutional neural network model is constructed and trained to obtain the product image analysis model which takes the product image as input and outputs the product image analysis result, wherein the product image analysis result includes a feature vector of the product image; Based on the text image data set, a text recognition model is constructed and trained to obtain a text image analysis model that takes the text image as input and outputs a text image analysis result, wherein the text image analysis result includes location information of the product text; Based on the traceability code image dataset, a target detection model is constructed and trained to obtain a traceability code image analysis model that takes the traceability code image as input and the traceability code image analysis result as output, wherein the traceability code image analysis result includes the location information of the product traceability code; The image big data analysis model is constructed based on the product image analysis model, the text image analysis model and the traceability code image analysis model.

5. The method according to claim 4, characterized in that The target product image analysis result includes a target product image feature vector of the product to be identified, the target text image analysis result includes the target product text position information of the product to be identified, and the target traceability code image analysis result includes the target product traceability code position information of the product to be identified; The first product traceability information, the second product traceability information and the third product traceability information are obtained by correspondingly querying from a product traceability information database based on the target product image analysis result, the target text image analysis result and the target traceability code image analysis result, respectively, including: Determine target category information of the product to be identified based on the product image dataset and the target product image feature vector, and obtain the first product traceability information by querying from the product traceability information database based on the target category information; Decoding the target product text position information to obtain the target text information of the product to be identified, and querying the product traceability information database based on the target text information to obtain the second product traceability information; The target decoder is used to decode the target product traceability code position information to obtain the target traceability code information of the product to be identified, and the third product traceability information is obtained by querying the product traceability information database based on the target traceability code information.

6. The method according to claim 5, characterized in that The determining the target category information of the product to be identified based on the product image dataset and the target product image feature vector includes: Vectorize each product image in the product image data set to obtain a feature vector of each product image and store it in a vector database to obtain a product image database; Calculating the similarity between the feature vector of the target product image and the feature vectors of each product image in the product image database to obtain a similarity ranking; Based on the similarity ranking, determining a similar image corresponding to the target product image; Based on the category information corresponding to the similar images, target category information of the product to be identified is determined.

7. The method according to claim 5, characterized in that The target decoder is a dual decoder, and the target decoder is used to decode the target product traceability code position information to obtain the target traceability code information of the product to be identified, including: Based on the target product traceability code position information, intercepting a target traceability code image area from the image to be identified according to a preset ratio, wherein the preset ratio is greater than 1; The target traceability code image area is decoded based on the dual decoder to obtain the target traceability code information of the product to be identified.

8. A product traceability information query device, characterized in that: The device comprises: An image acquisition module is used to acquire an image to be identified, wherein the image to be identified includes a target product image, a target text image, and a target traceability code image of the product to be identified; A result analysis module is used to input the image to be identified into a trained image big data analysis model, and obtain a target product image analysis result, a target text image analysis result, and a target traceability code image analysis result based on the product image analysis model, the text image analysis model, and the traceability code image analysis model in the image big data analysis model, respectively; An information query module, used to query and obtain first product traceability information, second product traceability information and third product traceability information from a product traceability information database based on the target product image analysis result, the target text image analysis result and the target traceability code image analysis result respectively; An information determination module is used to determine the target product traceability information of the commodity to be identified based on the first product traceability information, the second product traceability information and the third product traceability information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images