Product label correction method, system, equipment and medium

Abstract

The invention discloses a product label correction method, system and device and a storage medium. The method comprises the steps of obtaining a to-be-recognized product label picture; detecting a character deflection angle in the product label picture through a pre-trained image classification model to obtain a first deflection angle; performing target detection on the product label picture, and determining a contour area of an identification code contained in the product label picture; determining a second deflection angle according to the inclination angle of the contour area relative to the horizontal direction; removing the contour area from the product label picture, detecting field lines in the product label picture, and determining a third deflection angle according to the field lines; and determining a target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and correcting the product label picture based on the target deflection angle. According to the method, the product label correction accuracy can be effectively improved. The method can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a product label correction method, system, device and storage medium. Background Art

[0002] Product label images refer to the images of labels or logos attached to products. These label images usually contain basic product information, instructions for use, warning signs, ingredient lists, production dates, expiration dates, manufacturer information, etc. In e-commerce and retail industries, product label images play an important role in helping consumers understand product features and make purchasing decisions. In the relevant technical field, there are applications for automated recognition of product label images.

[0003] In order to realize the recognition of product label content, it is usually necessary to correct the angle of the product label image to a normal angle first, so that the recognition accuracy is relatively higher when performing content recognition. Some current technical solutions, when correcting product label images, are limited by the high complexity of the image content, and the correction effect is often unsatisfactory and the accuracy is low, which affects subsequent processing tasks.

[0004] In summary, the problems existing in related technologies need to be solved urgently. Summary of the invention

[0005] The purpose of this application is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, an object of embodiments of the present application is to provide a method, system, device and storage medium for correcting product labels.

[0007] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present application include:

[0008] In one aspect, an embodiment of the present application provides a method for correcting a product label, the method comprising:

[0009] Get the product label image to be identified;

[0010] The deflection angle of the text in the product label image is detected by a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image;

[0011] Performing target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image;

[0012] determining a second deflection angle according to an inclination angle of the contour area relative to a horizontal direction;

[0013] Eliminate the contour area from the product label image, detect field lines in the product label image, and determine a third deflection angle according to the field lines;

[0014] A target deflection angle is determined according to the first deflection angle, the second deflection angle, and the third deflection angle, and the product label image is corrected based on the target deflection angle.

[0015] In addition, the product label correction method according to the above embodiment of the present application may also have the following additional technical features:

[0016] Furthermore, in one embodiment of the present application, the image classification model is trained in the following manner:

[0017] Acquire multiple sample label images and label information corresponding to each of the sample label images; wherein the sample label images include text content, and the label information is used to record the actual deflection angle of the text content in the sample label images relative to the horizontal direction;

[0018] The deflection angle of the text in the sample label image is detected by the image classification model to be trained to obtain a fourth deflection angle; wherein the fourth deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the sample label image;

[0019] Determining a loss value of training according to the fourth deflection angle and the label information;

[0020] According to the loss value, the parameters of the image classification model are updated to obtain a trained image classification model.

[0021] Furthermore, in one embodiment of the present application, the identification code is a QR code or a barcode.

[0022] Further, in one embodiment of the present application, the performing target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image includes:

[0023] Input the product label image into the YOLOv5 target detection model;

[0024] Performing target detection on the product label image using the YOLOv5 target detection model to determine the region of interest of the identification code;

[0025] Performing binarization processing and expansion and corrosion processing on the region of interest;

[0026] The minimum bounding rectangle of the processed region of interest is detected, and the minimum bounding rectangle is determined as the contour region of the identification code.

[0027] Further, in one embodiment of the present application, the detecting the field lines in the product label image and determining the third deflection angle according to the field lines includes:

[0028] Detecting field lines in the product label image through Hough transform;

[0029] Detecting the sub-deflection angles of each of the field lines, clustering each of the sub-deflection angles using a preset clustering algorithm, and obtaining a first clustering category corresponding to each of the sub-deflection angles;

[0030] The third deflection angle is determined according to the first cluster category containing the largest number of the sub-deflection angles.

[0031] Further, in one embodiment of the present application, determining a target deflection angle according to the first deflection angle, the second deflection angle, and the third deflection angle includes:

[0032] detecting a difference between the first deflection angle, the second deflection angle, and the third deflection angle;

[0033] Comparing the difference with a preset threshold;

[0034] If the difference is less than or equal to the threshold, determining the target deflection angle according to an average of the first deflection angle, the second deflection angle, and the third deflection angle;

[0035] If the difference is greater than the threshold, the first deflection angle, the second deflection angle and the third deflection angle are clustered by a preset clustering algorithm, and the target deflection angle is determined according to the clustering result.

[0036] Furthermore, in one embodiment of the present application, the clustering algorithm is a k-means++ algorithm.

[0037] On the other hand, an embodiment of the present application provides a product label correction system, the system comprising:

[0038] An acquisition unit, used to acquire a product label image to be identified;

[0039] A first detection unit is used to detect the deflection angle of the text in the product label image by using a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image;

[0040] A second detection unit, configured to perform target detection on the product label image to determine a contour area of ​​the identification code contained in the product label image;

[0041] a processing unit, configured to determine a second deflection angle according to an inclination angle of the contour area relative to a horizontal direction;

[0042] A third detection unit is used to remove the contour area from the product label image, detect the field lines in the product label image, and determine a third deflection angle according to the field lines;

[0043] An execution unit is used to determine a target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and correct the product label image based on the target deflection angle.

[0044] On the other hand, an embodiment of the present application provides an electronic device, including:

[0045] at least one processor;

[0046] at least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned product label correction method.

[0048] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned product label correction method.

[0049] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0050] The correction method, system, device and storage medium of product labels disclosed in the embodiments of the present application, the method obtains a product label image to be identified; detects the deflection angle of the text in the product label image through a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is the deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image; performs target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image; determines the second deflection angle according to the inclination of the contour area relative to the horizontal direction; removes the contour area from the product label image, detects the field lines in the product label image, and determines the third deflection angle according to the field lines; determines the target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and corrects the product label image based on the target deflection angle. This method determines the target deflection angle by evaluating and detecting the deflection angle of the product label image in multiple dimensions, and then corrects the product label image according to the target deflection angle, which can effectively improve the accuracy of product label correction and facilitate subsequent accurate identification of its content. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present application. For technical personnel in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A schematic diagram of an implementation environment of a product label correction method provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a process for correcting a product label provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a training process of an image classification model provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of a process for determining a contour area of ​​an identification code provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of a process for determining a third deflection angle according to field lines provided in an embodiment of the present application;

[0057] Figure 6A schematic diagram of a process for determining a target deflection angle provided in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of a specific implementation process of a product label correction method provided in an embodiment of the present application;

[0059] Figure 8 A schematic diagram of a device module for implementing a product label correction method provided in an embodiment of the present application;

[0060] Fig. 9 A structural schematic diagram of a product label correction system provided in an embodiment of the present application;

[0061] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0063] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0065] 1) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0066] 2) Image recognition is an important branch of artificial intelligence (AI), which involves the technology of detecting and identifying objects, people, scenes, actions and other elements from digital images or videos. The goal of image recognition is to enable computers to "see" and "understand" visual information like humans, so as to achieve intelligent decision-making and operation.

[0067] 3) Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning (deep learning) usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0068] 4) Deep learning is a method of machine learning and a branch of artificial intelligence. It imitates the neural network structure of the human brain and learns and extracts data features through a multi-level neural network structure to achieve modeling and prediction of complex data. Deep learning usually uses a large amount of data and computing resources for training to improve the accuracy and generalization ability of the model. Deep learning has achieved many successful applications in image recognition, speech recognition, natural language processing and other fields, and has become one of the hot technologies in the field of artificial intelligence.

[0069] 5) Clustering is an unsupervised learning method that aims to divide a set of objects into several groups (called "clusters") so that objects within the same cluster are more similar, while objects in different clusters are less similar. Clustering algorithms attempt to discover the inherent structure of data, thereby grouping or classifying data without pre-defining the category to which each object belongs.

[0070] Product label images refer to the images of labels or logos attached to products. These label images usually contain basic product information, instructions for use, warning signs, ingredient lists, production dates, expiration dates, manufacturer information, etc. In e-commerce and retail industries, product label images play an important role in helping consumers understand product features and make purchasing decisions. In the relevant technical field, there are applications for automated recognition of product label images.

[0071] At present, in the field of product label recognition in natural scenes, in order to realize the recognition of product label content, the following steps are usually required: 1. Image preprocessing, including image angle correction, image enhancement, etc.; 2. Positioning of field frame, identification code (QR code, barcode, etc.); 3. Text recognition and identification code recognition in field frame. In the above processing process, the result of each step directly affects the accuracy of the result obtained in the next step, and ultimately affects the overall recognition performance of the product label content recognition system.

[0072] When identifying the original product labels, it will be found that some product label images are rotated or tilted. The tilt or rotation problem of the product label image will have a great impact on the positioning of the product label field box and the identification code; the failure to obtain the correct field and barcode position will naturally greatly affect the accuracy of text and identification code recognition. Therefore, how to accurately correct the angle of the original product label image is an important topic in the product label content recognition system in natural scenes. The types of product label images are relatively complex, including a small number of fields, non-standard rectangular fields, barcodes, QR codes, etc. Some current technical solutions, when correcting product label images, are limited by the high complexity of the image content, and the correction effect is often not ideal and the accuracy is low, which affects subsequent processing tasks.

[0073] In view of this, a product label correction method is provided in an embodiment of the present application. The method obtains a product label image to be identified; detects the text deflection angle in the product label image through a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is the deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image; performs target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image; determines the second deflection angle according to the inclination of the contour area relative to the horizontal direction; removes the contour area from the product label image, detects the field lines in the product label image, and determines the third deflection angle according to the field lines; determines the target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and corrects the product label image based on the target deflection angle. The method determines the target deflection angle by evaluating and detecting the deflection angle of the product label image in multiple dimensions, and then corrects the product label image according to the target deflection angle, which can effectively improve the accuracy of product label correction and facilitate subsequent accurate recognition of its content.

[0074] Please refer to Figure 1 , Figure 1 The schematic diagram of the implementation environment of a product label correction method provided in the embodiment of the present application is shown. In the implementation environment, the main hardware and software entities involved include a terminal device 110 and a backend server 120. The terminal device 110 and the backend server 120 are connected in communication.

[0075] Specifically, the product label correction method provided in the embodiment of the present application can be executed alone on the terminal device 110 side, or can be executed alone on the background server 120 side, or can be executed based on data interaction between the terminal device 110 and the background server 120.

[0076] The terminal device 110 of the above embodiment may include a mobile phone, a computer, a smart wearable device, a PDA device, an intelligent voice interaction device, a vehicle terminal, etc., but is not limited thereto. The backend server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0077] The terminal device 110 and the backend server 120 may establish a communication connection via a wireless network or a wired network. The wireless network or wired network uses standard communication technology and / or protocols, and the network may be set to the Internet or any other network, such as but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0078] Of course, it is understandable that Figure 1 The implementation environment in the embodiment of the present application is only some optional application scenarios of the product label correction method provided in the embodiment of the present application, and the actual application is not fixed. Figure 1 The hardware and software environment shown.

[0079] Below, in combination with the introduction of the aforementioned implementation environment, a product label correction method provided in an embodiment of the present application is introduced and explained.

[0080] Please refer to Figure 2 , Figure 2 : is a schematic diagram of a product label correction method provided in an embodiment of the present application, and the product label correction method includes but is not limited to:

[0081] Step 210: Obtain a product label image to be identified;

[0082] Step 220: Detect the deflection angle of the text in the product label image by using a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image;

[0083] Step 230: Perform target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image;

[0084] Step 240: determining a second deflection angle according to the inclination angle of the contour area relative to the horizontal direction;

[0085] Step 250: remove the contour area from the product label image, detect the field lines in the product label image, and determine a third deflection angle according to the field lines;

[0086] Step 260: Determine a target deflection angle according to the first deflection angle, the second deflection angle, and the third deflection angle, and correct the product label image based on the target deflection angle.

[0087] In an embodiment of the present application, a product label correction method is provided, which can effectively improve the accuracy of product label correction and facilitate subsequent accurate identification of its content.

[0088] Specifically, in an embodiment of the present application, when executing the product label correction method, first, a product label image to be identified can be obtained. For example, in some embodiments, the product label image can be acquired by collecting and acquiring through relevant camera equipment; in some embodiments, the product label image can also be acquired through other input sources, and the present application does not impose any restrictions on this.

[0089] It should be noted that in the embodiments of the present application, after obtaining the product label image, in some cases, the product label image can be subjected to corresponding preprocessing operations. For example, in some embodiments, the product label image can be resized, that is, the product label image can be adjusted to a predetermined size for subsequent processing. In some embodiments, the product label image can be cropped to remove parts of the image that are not related to the task to reduce interference factors. In some embodiments, the product label image can also be subjected to denoising and other operations, such as using a filter (such as a Gaussian filter) to remove noise in the image to improve the image quality, thereby improving the correction and recognition effects of the image.

[0090] In an embodiment of the present application, for a product label image, first, a pre-trained image classification model can be used to detect the text deflection angle therein to obtain a prediction result of a deflection angle, which is recorded as a first deflection angle in an embodiment of the present application. Here, the image classification model can be trained to be suitable for the task of detecting the text deflection angle in the product label image. For example, for example, the category output by the image classification model can be pre-set as the angle value of the text deflection angle, such as 0, 10, 20...360, a total of 36 classification categories. Of course, in some other embodiments, the category of the angle value output by the image classification model can be further subdivided or flexibly adjusted according to demand, and the present application does not limit this. The present application does not limit the specific type and network structure of the image classification model, for example, it can adopt a resnet50 model or a convolutional neural network CNN.

[0091] It should be noted that, to facilitate subsequent processing, the first deflection angle in the embodiment of the present application can be the deflection angle of the text in the predicted product label image relative to the horizontal direction. Similarly, other deflection angles appearing later can also be deflection angles relative to the horizontal direction. This application does not impose any restrictions on this.

[0092] In step 230 of the embodiment of the present application, target detection can also be performed on the product label image. The purpose of target detection is to detect and locate the identification code in the image. In the embodiment of the present application, the type of identification code may include but is not limited to barcodes, QR codes, etc. This application does not limit the specific type of algorithm used for target detection. For example, in some embodiments, target detection algorithms such as YOLO, Faster R-CNN, etc. can be used to input the product label image, and the model can output the contour area of ​​the identification code.

[0093] In the embodiment of the present application, the inclination of the contour area of ​​the identification code relative to the horizontal direction can be determined, so as to determine the deflection angle in the dimension of the identification code, which is recorded as the second deflection angle in the embodiment of the present application. It should be noted that there may be multiple identification codes in the product label image, so when determining the second deflection angle based on the inclination of the contour area relative to the horizontal direction, the number of the obtained second deflection angles can be multiple, and the present application does not limit the specific number.

[0094] In step 240 of the embodiment of the present application, the contour area can be removed from the product label image, and then the field lines in the product label image can be detected. According to the field lines, the tilt deflection of the product label image can also be evaluated. In the embodiment of the present application, the deflection angle can be determined according to the field lines, which is recorded as the third deflection angle. Similarly, it is easy to understand that in the product label image, the number of fields may be large. Therefore, when determining the third deflection angle according to the field lines, the number of the third deflection angles obtained may also be multiple, and the present application does not limit the specific number thereof.

[0095] After obtaining the first deflection angle, the second deflection angle, and the third deflection angle, the target deflection angle can be determined based on these deflection angles. It can be understood that in the embodiment of the present application, the target deflection angle is determined based on the deflection conditions in multiple dimensions, and can make full use of various types of information in the product label image, which can effectively improve the accuracy of the product label angle correction. After the target deflection angle is determined, the product label image can be corrected based on the target deflection angle. Specifically, the product label image can be rotated in the opposite direction according to the value of the target deflection angle, so that the product label image returns to its normal position.

[0096] It can be understood that the technical solution in the embodiment of the present application obtains a product label image to be identified; detects the deflection angle of the text in the product label image through a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is the deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image; performs target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image; determines the second deflection angle according to the inclination of the contour area relative to the horizontal direction; removes the contour area from the product label image, detects the field lines in the product label image, and determines the third deflection angle according to the field lines; determines the target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and corrects the product label image based on the target deflection angle. This method determines the target deflection angle by evaluating and detecting the deflection angle of the product label image in multiple dimensions, and then corrects the product label image according to the target deflection angle, which can effectively improve the accuracy of product label correction and facilitate subsequent accurate recognition of its content.

[0097] Specifically, in some embodiments, referring to Figure 3 , the image classification model is trained in the following way:

[0098] Acquire multiple sample label images and label information corresponding to each of the sample label images; wherein the sample label images include text content, and the label information is used to record the actual deflection angle of the text content in the sample label images relative to the horizontal direction;

[0099] The deflection angle of the text in the sample label image is detected by the image classification model to be trained to obtain a fourth deflection angle; wherein the fourth deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the sample label image;

[0100] Determining a loss value of training according to the fourth deflection angle and the label information;

[0101] According to the loss value, the parameters of the image classification model are updated to obtain a trained image classification model.

[0102] In an embodiment of the present application, the image classification model needs to be trained before use to improve the application performance of the model as much as possible. Specifically, when training the image classification model, some sample label pictures and label information corresponding to the sample label pictures can be obtained in batches. Here, the sample label picture can be a picture related to or similar to the aforementioned product label picture, which also contains text content, and the label information can be used to record the actual deflection angle of the text content in the sample label picture relative to the horizontal direction. Then, for each sample label picture, the image classification model can be used to perform reasoning analysis on it to obtain the predicted result of the text deflection angle corresponding to the sample label picture. In an embodiment of the present application, it is recorded as the fourth deflection angle.

[0103] It is understandable that in the embodiment of the present application, if the prediction effect of the image classification model is good, then the fourth deflection angle and label information corresponding to the sample label image should be relatively close. Therefore, in the embodiment of the present application, the accuracy of the image classification model prediction can be determined based on the fourth deflection angle and label information. Specifically, the deviation between the fourth deflection angle and the label information can be determined to obtain the loss value predicted by the image classification model. After obtaining the loss value, the prediction accuracy of the image classification model can be evaluated according to the size of the loss value, so as to perform back-propagation training on the image classification model and update the relevant parameters inside it.

[0104] Specifically, for a machine learning model (or a deep learning model), the accuracy of its prediction can be measured by a loss function, which is defined on a single training data and is used to measure the prediction error of a training data. Specifically, the loss value of the training data is determined by the label of a single training data and the prediction result of the model for the training data. During actual training, a training data set has a lot of training data, so a cost function is generally used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the prediction error of all training data, which can better measure the prediction effect of the model. For a general model, based on the aforementioned cost function, plus a regularization term that measures the complexity of the model, it can be used as the objective function of the training, and the loss value of the entire training data set can be calculated based on the objective function. There are many types of commonly used loss functions, such as 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, cross entropy loss function, etc., which can all be used as loss functions of machine learning models, which will not be described one by one here. In the embodiment of the present application, any loss function can be selected to determine the loss value, so as to update the parameters of the image classification model.

[0105] In the embodiment of the present application, the parameter update of the image classification model can be performed in a loop iteration manner. That is, after updating the parameters of a round of image classification model, the image classification model with updated parameters is continued to be used for prediction, a new loss value is determined, and then the parameters of the image classification model are updated again. This cycle is repeated until the pre-set training end condition is met, and the training can be considered completed, and a trained image classification model is obtained.

[0106] In the embodiments of the present application, the conditions for the end of training can be flexibly set according to the needs. For example, in some embodiments, the target round of training loop iteration can be set as the condition for the end of training. When the update round of the parameters of the image classification model reaches the target round, the training can be considered to be completed; in some embodiments, the difference threshold of the loss value obtained in two adjacent training processes can be set as the condition for the end of training. After updating the parameters of a round of image classification model, the absolute value of the difference between the loss value obtained in this round of training and the loss value obtained in the previous round of training is calculated. If the absolute value of the difference is greater than the set difference threshold, the iterative training continues; if the absolute value of the difference is less than or equal to the set difference threshold, the training can be considered to be completed. Of course, the above is only an exemplary introduction to some optional training end condition setting methods in the embodiments of the present application, and does not mean to limit the situation during actual implementation.

[0107] For details, please refer to Figure 4 , the performing target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image includes:

[0108] Input the product label image into the YOLOv5 target detection model;

[0109] Performing target detection on the product label image using the YOLOv5 target detection model to determine the region of interest of the identification code;

[0110] Performing binarization processing and expansion and corrosion processing on the region of interest;

[0111] The minimum bounding rectangle of the processed region of interest is detected, and the minimum bounding rectangle is determined as the contour region of the identification code.

[0112] In an embodiment of the present application, when detecting the contour area of ​​the identification code contained in the product label image, the YOLOv5 target detection model can be used. YOLOv5 (You Only Look Once v5) is a target detection model based on deep learning, which can quickly and accurately detect multiple objects in the image. In an embodiment of the present application, the product label image is input into the pre-trained YOLOv5 model, which extracts image features through a convolutional neural network (CNN) and generates a bounding box and category probability to identify the target in the image.

[0113] Specifically, in an embodiment of the present application, when performing target detection through the YOLOv5 target detection model, the YOLOv5 model will output one or more bounding boxes, which represent the areas in the image that may contain identification codes. Each bounding box usually includes four coordinates (x, y, w, h), which respectively represent the center point coordinates and width and height of the bounding box. In addition, the model will also output a confidence score, indicating the probability of the presence of the identification code in the bounding box. In an embodiment of the present application, the region of interest of the identification code can be determined according to the confidence score.

[0114] For each region of interest, it can be subjected to binarization and dilation and erosion. Among them, binarization is to convert the image into an image with only black and white colors. Specifically, a threshold can be selected to set the pixels in the image with a value greater than the threshold to white (255), and the pixels less than or equal to the threshold to black (0). This operation helps to remove background noise and highlight the characteristics of the identification code. Dilation and erosion are morphological operations used to improve the quality of the binary image. The dilation operation will expand the white area in the image, while the erosion operation will shrink the white area. Usually, the erosion operation is first performed to remove small noise points, and then the dilation operation is performed to restore the eroded part, so that the outline of the identification code is clearer. After binarization and dilation and erosion processing, the identification code area in the image should be very clear. Next, the minimum bounding rectangle of each processed region of interest can be determined, and the minimum bounding rectangle of the region of interest can be determined as the outline area of ​​the identification code for subsequent recognition and processing.

[0115] Specifically, in some embodiments, referring to Figure 5 , the detecting the field lines in the product label image and determining the third deflection angle according to the field lines comprises:

[0116] Detecting field lines in the product label image through Hough transform;

[0117] Detecting the sub-deflection angles of each of the field lines, clustering each of the sub-deflection angles using a preset clustering algorithm, and obtaining a first clustering category corresponding to each of the sub-deflection angles;

[0118] The third deflection angle is determined according to the first cluster category containing the largest number of the sub-deflection angles.

[0119] In the embodiment of the present application, when detecting the field lines in the product label image to determine the third deflection angle, in some embodiments, the Hough transform algorithm can be used. Hough transform is a technique for finding specific shapes (such as straight lines, circles, etc.) in an image. For straight line detection, the basic principle of Hough transform is to convert points in the image space into curves in the parameter space, and the intersection of these curves corresponds to the straight lines in the original image. In the embodiment of the present application, the field lines in the product label image can be detected by Hough transform, and then the deflection angles of each field line can be detected. In the embodiment of the present application, it is recorded as a sub-deflection angle. After obtaining the sub-deflection angles of each field line, they can be clustered. Commonly used clustering algorithms include K-means, DBSCAN, etc. In the embodiment of the present application, the k-means++ algorithm can be used. The K-means++ algorithm is an improved K-means clustering algorithm that aims to solve the problem that the random selection of the initial clustering center in the traditional K-means algorithm may lead to slow convergence and poor clustering effect. K-means++ selects the initial clustering center in a more intelligent way, thereby improving the performance and stability of the algorithm.

[0120] In the embodiment of the present application, after clustering the sub-deflection angles, the first clustering category corresponding to each sub-deflection angle can be obtained. It is easy to understand that, generally speaking, the deflection of the field lines is relatively consistent, and the sub-deflection angles corresponding to most field lines are relatively close, so most of the sub-deflection angles will be in a first clustering category. In the embodiment of the present application, the third deflection angle can be determined according to the first clustering category with the largest number of sub-deflection angles.

[0121] Specifically, in some embodiments, referring to Figure 6 , determining a target deflection angle according to the first deflection angle, the second deflection angle, and the third deflection angle, comprises:

[0122] detecting a difference between the first deflection angle, the second deflection angle, and the third deflection angle;

[0123] Comparing the difference with a preset threshold;

[0124] If the difference is less than or equal to the threshold, determining the target deflection angle according to an average of the first deflection angle, the second deflection angle, and the third deflection angle;

[0125] If the difference is greater than the threshold, the first deflection angle, the second deflection angle and the third deflection angle are clustered by a preset clustering algorithm, and the target deflection angle is determined according to the clustering result.

[0126] In the embodiment of the present application, when determining the target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, clustering can also be used. Specifically, in the embodiment of the present application, the difference between the first deflection angle, the second deflection angle and the third deflection angle can be detected first, for example, the difference can be made between the two to obtain the corresponding difference. Then, the difference can be compared with a preset threshold.

[0127] In some cases, the differences may be less than or equal to the threshold, which means that the deflection angles determined in each dimension are relatively close. At this time, the average of the first deflection angle, the second deflection angle and the third deflection angle can be directly calculated, and the obtained average can be determined as the target deflection angle.

[0128] In some cases, one or some differences may be greater than a threshold value, which means that there are certain differences in the deflection angles determined in each dimension. In this case, the first deflection angle, the second deflection angle, and the third deflection angle can be clustered, and the target deflection angle can be determined based on the clustering results. For example, the clustering category with the most deflection angles after clustering can be determined as the target category, and then the target deflection angle is determined based on the mean of the deflection angles in the target category. Here, the clustering algorithm used can also be the aforementioned K-means++ algorithm, and this application does not limit this.

[0129] Reference Figure 7 and Figure 8 , Figure 7 A schematic diagram of a specific implementation process of a product label correction method provided in an embodiment of the present application is shown. Figure 8 It is a schematic diagram of a device module for implementing the correction method of the product label.

[0130] In the embodiment of the present application, the specific implementation process may include:

[0131] Step 1: Product label input. Input the product label image to be identified.

[0132] Step 2: Image classification resnet50 model and angle calculation based on text features. According to the text field features in the product label image in the natural scene, train the resnet50 image classification model with 36 types of angle classifications such as 0, 10, and 20 (it can be further subdivided, the more types, the finer). Input the product label image to be identified into the model, and the model will classify it into one of the 36 types of pre-trained angle images, so as to obtain the correction angle (i.e., the first deflection angle) of the current product label image based on text features.

[0133] Step 3: YOLOv5 model and angle calculation for target detection based on barcode features. According to the features of QR codes and barcodes in product label images in natural scenes, the YOLOv5 target detection model for both QR codes and barcodes is trained. Input the product label image to be identified into the model, and the model will detect all QR code and barcode areas. After binarization in this area, a certain degree of expansion and corrosion is performed to obtain a new rectangle. The minimum outer bounding box of this rectangle is calculated, and the inclination of the barcode in this area relative to the horizontal direction can be obtained based on the coordinates of this outline box. Therefore, multiple correction angles (i.e., the second deflection angle) can be obtained based on the features of the QR code and barcode in the product label image to be identified.

[0134] Step 4: Hough transform, k-means clustering and angle calculation based on line features. After removing the QR code and barcode areas from the product label image to be identified, calculate the correction angle based on the text line features. Through Hough transform, detect the field lines in the product label image to be identified, count the angles between the lines and the horizontal direction, and perform k-means++ mean clustering with k=3 (that is, the final results are divided into 3 categories, correct inclination, large inclination, and small inclination) for all the angles between the lines and the horizontal direction (positive direction). The angle value of the cluster center point of the category with the most points is taken as the correction angle based on the line (that is, the third deflection angle).

[0135] Step 5: Multi-angle fusion. After calculating the four types of correction angles according to the above steps, calculate the difference between each correction angle. When the maximum difference exceeds 5 degrees, perform k-means++ mean clustering with k=3 (correct inclination, large inclination, small inclination) on the four types of correction angles. According to the final correction angle (i.e., the target deflection angle), perform angle correction on the product label image to be identified.

[0136] Reference Fig. 9 In an embodiment of the present application, a product label correction system is also provided, including:

[0137] An acquisition unit 910 is used to acquire a product label image to be identified;

[0138] A first detection unit 920 is used to detect the deflection angle of the text in the product label image by using a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image;

[0139] A second detection unit 930 is used to perform target detection on the product label image to determine a contour area of ​​the identification code contained in the product label image;

[0140] A processing unit 940 is configured to determine a second deflection angle according to an inclination angle of the contour area relative to a horizontal direction;

[0141] A third detection unit 950 is used to remove the contour area from the product label image, detect the field lines in the product label image, and determine a third deflection angle according to the field lines;

[0142] The execution unit 960 is used to determine a target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and correct the product label image based on the target deflection angle.

[0143] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0144] Reference Fig.10 , an embodiment of the present application provides an electronic device, including:

[0145] at least one processor 1010;

[0146] At least one memory 1020, used to store at least one program;

[0147] When at least one program is executed by at least one processor 1010, at least one processor 1010 implements the above-mentioned product label correction method.

[0148] Similarly, the contents of the above method embodiments are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0149] The embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the processor 1010 is stored. When the program executable by the processor 1010 is executed by the processor 1010, it is used to execute the above-mentioned product label correction method.

[0150] Similarly, the contents of the above method embodiments are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0151] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0152] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0153] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0154] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0155] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0156] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0157] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0158] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0159] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for correcting a product label, characterized in that: The method comprises: Get the product label image to be identified; The deflection angle of the text in the product label image is detected by a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image; Performing target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image; determining a second deflection angle according to an inclination angle of the contour area relative to a horizontal direction; Eliminate the contour area from the product label image, detect field lines in the product label image, and determine a third deflection angle according to the field lines; A target deflection angle is determined according to the first deflection angle, the second deflection angle, and the third deflection angle, and the product label image is corrected based on the target deflection angle.

2. A product label correction method according to claim 1, characterized in that: The image classification model is trained in the following way: Acquire multiple sample label images and label information corresponding to each of the sample label images; wherein the sample label images include text content, and the label information is used to record the actual deflection angle of the text content in the sample label images relative to the horizontal direction; Detecting the deflection angle of the text in the sample label image by the image classification model to be trained to obtain a fourth deflection angle; wherein the fourth deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the sample label image; Determining a loss value of training according to the fourth deflection angle and the label information; According to the loss value, the parameters of the image classification model are updated to obtain a trained image classification model.

3. A product label correction method according to claim 1, characterized in that: The identification code is a QR code or a bar code.

4. A product label correction method according to claim 3, characterized in that: The performing target detection on the product label image to determine the contour area of ​​the identification code contained in the product label image includes: Input the product label image into the YOLOv5 target detection model; Performing target detection on the product label image using the YOLOv5 target detection model to determine the region of interest of the identification code; Performing binarization processing and expansion and corrosion processing on the region of interest; The minimum bounding rectangle of the processed region of interest is detected, and the minimum bounding rectangle is determined as the contour region of the identification code.

5. A product label correction method according to claim 1, characterized in that: The detecting the field lines in the product label image and determining the third deflection angle according to the field lines includes: Detecting field lines in the product label image through Hough transform; Detecting the sub-deflection angles of each of the field lines, clustering each of the sub-deflection angles using a preset clustering algorithm, and obtaining a first clustering category corresponding to each of the sub-deflection angles; The third deflection angle is determined according to the first cluster category containing the largest number of the sub-deflection angles.

6. A product label correction method according to claim 1, characterized in that: The determining a target deflection angle according to the first deflection angle, the second deflection angle, and the third deflection angle comprises: detecting a difference between the first deflection angle, the second deflection angle, and the third deflection angle; Comparing the difference with a preset threshold; If the difference is less than or equal to the threshold, determining the target deflection angle according to an average of the first deflection angle, the second deflection angle, and the third deflection angle; If the difference is greater than the threshold, the first deflection angle, the second deflection angle and the third deflection angle are clustered by a preset clustering algorithm, and the target deflection angle is determined according to the clustering result.

7. A product label correction method according to claim 5 or 6, characterized in that: The clustering algorithm is the k-means++ algorithm.

8. A product label correction system, characterized in that: The system comprises: An acquisition unit, used to acquire a product label image to be identified; A first detection unit is used to detect the deflection angle of the text in the product label image by using a pre-trained image classification model to obtain a first deflection angle; wherein the first deflection angle is a deflection angle relative to the horizontal direction predicted by the image classification model based on the text in the product label image; A second detection unit, configured to perform target detection on the product label image to determine a contour area of ​​the identification code contained in the product label image; a processing unit, configured to determine a second deflection angle according to an inclination angle of the contour area relative to a horizontal direction; A third detection unit is used to remove the contour area from the product label image, detect the field lines in the product label image, and determine a third deflection angle according to the field lines; An execution unit is used to determine a target deflection angle according to the first deflection angle, the second deflection angle and the third deflection angle, and correct the product label image based on the target deflection angle.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a product label correction method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to implement a product label correction method as described in any one of claims 1-7 when executed by the processor.

Images