Label identification method and device, electronic equipment, medium and computer program product

Abstract

The embodiment of the invention discloses a label identification method and device, electronic equipment, a medium and a computer program product, and the method comprises the steps: processing an image of a label, and obtaining a first feature map corresponding to the image; according to a global average value of a first channel dimension of the first feature map, obtaining a feature value of the first channel dimension, the feature value representing an offset degree of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map; determining the weight of the first channel dimension based on the feature value of the first channel dimension; the weight of the first channel dimension is negatively correlated with the characteristic value of the first channel dimension; obtaining a processed feature map based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension; and outputting the information of the tag according to the processed feature map.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence, and in particular relates to a tag recognition method, device, electronic device, medium and computer program product. Background Art

[0002] Currently, most methods for label recognition of devices to be identified are based on traditional signal processing or traditional computer vision key point detection methods. These methods are usually unable to identify the precise position of the label to be identified, or are highly dependent on the pixels and resolution of the camera that captures the label image, resulting in poor label position recognition results. Summary of the invention

[0003] Embodiments of the present application provide a tag identification method, device, electronic device, medium, and computer program product.

[0004] The present application provides a tag identification method, the method comprising:

[0005] Processing the labeled image to obtain a first feature map corresponding to the image;

[0006] Obtaining a feature value of the first channel dimension according to a global average value of the first channel dimension of the first feature map, wherein the feature value represents a degree of deviation of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map;

[0007] Determining a weight of the first channel dimension based on the eigenvalue of the first channel dimension; the weight of the first channel dimension is negatively correlated with the eigenvalue of the first channel dimension;

[0008] Based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension, a processed feature map is obtained; and according to the processed feature map, information of the label is output.

[0009] In some embodiments, the eigenvalue is the variance corresponding to each value in the first channel dimension.

[0010] It can be seen that by calculating the variance corresponding to each value in the first channel dimension, the degree of deviation of each value in the first channel dimension relative to the global average value can be obtained. Since the first channel dimension is part of the feature map that constitutes the label image, the weight of each first channel dimension is determined by the variance, and the processed feature map corresponding to the label is obtained by weighted processing of multiple first channel dimensions, which is beneficial to more accurately reflect the characteristics of the label image through the processed feature map, and further beneficial to obtain accurate label information based on the processed feature map.

[0011] In some embodiments, processing the image of the label to obtain a first feature map corresponding to the image includes: processing the image of the label to obtain a second feature map; calculating the first value in the second feature map to obtain the first feature map corresponding to the image; wherein the first value is each value in each channel of the second feature map; the first feature map includes multiple second values, and the second values ​​correspond to the first values ​​one by one; when the first value is greater than 0, the second value is greater than or equal to the first value; when the first value is less than 0, the second value is less than or equal to 0.

[0012] It can be seen that since the size of each numerical value in each channel of the feature map can reflect the edge features of the image, the positive numerical value in the feature map corresponds to a strong edge, while the negative numerical value corresponds to a weak edge or background area. Through the method provided in this embodiment, when the first numerical value is greater than 0, the features of the first numerical value can be retained or amplified; when the first numerical value is less than 0, the features of the first numerical value less than 0 can be removed, or part of the features of the first numerical value can be retained. In other words, through the method provided in this embodiment, the features of the first numerical value greater than 0 can be made more obvious, which is conducive to more in-depth extraction of the edge features of the label image.

[0013] In some embodiments, the calculating the first value in the second characteristic map includes: calculating the first value at least by using a hyperbolic tangent function.

[0014] It can be seen that calculating the first value by using the hyperbolic tangent function to obtain the second value is conducive to making the edge feature more prominent and is conducive to more in-depth extraction of the edge feature of the label image.

[0015] In some embodiments, the independent variable of the hyperbolic tangent function is positively correlated with the first value.

[0016] It can be seen that processing the first value by the method provided in this embodiment is conducive to further extracting the edge features of the label image.

[0017] In some embodiments, before processing the labeled image, the method further includes: performing data enhancement on the training image, performing model training based on the data enhanced training image, and obtaining a label recognition model; the label recognition model is used to process the labeled image and output label information; wherein, performing data enhancement on the training image includes: increasing the shadow of the training image, and / or increasing the brightness of the training image.

[0018] It can be seen that by performing data enhancement on the training images and training the model based on the data enhanced training images, it is beneficial to obtain a label recognition model with stronger recognition ability. Furthermore, by adding shadows to the training images and / or increasing the brightness of the training images before training the model, it is beneficial for the trained model to have stronger recognition ability for shadow images or images with enhanced brightness.

[0019] In some embodiments, increasing the shadow of the training image includes: multiplying a mask function by the pixel value of the training image to increase the shadow of the processed training image; the value range of the mask function is greater than 0 and less than 1; the mask function is positively correlated with the pixel value before modification.

[0020] It can be seen that by processing the pixel value of the training image with a mask function whose value range is between 0 and 1, the pixel value of the training image can be reduced and the shadow of the training image can be increased.

[0021] In some embodiments, improving the brightness of the training image includes: adding a brightening function to the pixel value of the training image to improve the brightness of the processed training image; the brightening function is negatively correlated with the pixel value before modification.

[0022] It can be seen that by processing the pixel value through the brightening function, the pixel value of the training image can be increased and the brightness of the training image can be improved. By setting the brightening function to be negatively correlated with the pixel value before modification, it is beneficial to obtain a better brightness improvement effect for images with higher grayscale values.

[0023] The present application also provides a tag identification device, the device comprising:

[0024] A first processing module, used to process the label image to obtain a first feature map corresponding to the image;

[0025] A second processing module is configured to obtain a feature value of the first channel dimension according to a global average value of the first channel dimension of the first feature map, wherein the feature value represents a degree of deviation of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map; based on the feature value of the first channel dimension, a weight of the first channel dimension is determined; and the weight of the first channel dimension is negatively correlated with the feature value of the first channel dimension;

[0026] An output module is used to obtain a processed feature map based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension; and output the label information according to the processed feature map.

[0027] An embodiment of the present application provides an electronic device, the electronic device comprising a processor and a memory for storing a computer program that can be run on the processor; wherein:

[0028] The processor is used to run the computer program to execute any one of the above-mentioned tag identification methods.

[0029] An embodiment of the present application provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned tag identification methods is implemented.

[0030] An embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned tag identification methods when executed by a processor.

[0031] The embodiments of the present application provide a label recognition method, device, electronic device, medium and computer program product. Through the label recognition method provided in the embodiments of the present application, the weight of the first channel dimension can be determined by the degree of deviation of each value in the first channel dimension relative to the global average value, and a processed feature map is obtained based on the value of the first channel dimension and the weight of the first channel dimension. The processed feature map can more accurately and significantly reflect the edge features of the label image, and further based on the processed feature map, more accurate label information can be output. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flow chart of a tag identification method provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of a function distribution curve provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of an improved model structure provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of the structure of a device label convolution module provided in an embodiment of the present application;

[0036] Figure 5 A processing flow chart of a label edge feature attention module provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of a training image set provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of another training image set provided in an embodiment of the present application;

[0039] Figure 8A schematic diagram of the structure of a tag identification device provided in an embodiment of the present application;

[0040] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] High-precision device label positioning and recognition is an indispensable part of the entire label recognition task, and is the foundation and key to the entire recognition process. In the device recognition task, the first task is to accurately identify the location information of the label. Only through high-precision label positioning and recognition can we ensure that the label image can be accurately and completely captured. If the positioning and recognition is inaccurate, the label image may be missing or blurred, which will directly affect the subsequent effect of the entire label recognition.

[0042] After successfully identifying the location information of the label, the next step is to capture the image of the area where the label is located. This process requires that the captured image not only contains complete label information, but also ensures the clarity and accuracy of the image. These factors directly determine the success or failure of the subsequent optical character recognition (OCR) operation.

[0043] After capturing a clear and complete label image, applying OCR technology to identify the label content is a crucial step. OCR technology can accurately extract text information from the image and convert it into an editable text format. This process relies on high-precision label positioning and image capture in the early stage, and any errors in positioning and capture will affect the accuracy of OCR recognition results.

[0044] Therefore, high-precision equipment tag positioning and identification is not only the starting point of the entire tag identification task, but also the premise to ensure the efficiency and accuracy of the entire tag identification process. It plays a decisive role in the accuracy and efficiency of equipment management, data recording and information processing. In many fields such as industry, logistics, and medical care, accurate tag positioning and identification provides solid support for information and intelligent management, and significantly improves the operational efficiency and management level of various industries. It can be said that without high-precision equipment tag positioning and identification, the entire tag identification task will be difficult to achieve its intended goal.

[0045] In a related technology, a method for positioning and identifying a label is provided: based on the label position information recorded in a preset configuration file, a first label image corresponding to the label position information in the image to be detected of the target device is extracted, and a template label image corresponding to the target device is obtained; a mask label image is determined based on the first label image and the template label image; the template label image is rotated in sequence according to a preset angle with the specified position of the template label image as the rotation center; for each template label image rotated by a preset angle, the mask label image is traversed according to the template label image, and the matching degree between the template label image and the corresponding area image of the template label image in the mask label image after each traversal is calculated; a target matching angle is determined according to the matching degree, and the target label position in the mask label image is determined based on the target matching angle; and whether the device label is qualified is determined according to the target label position.

[0046] In another related technology, a method for detecting labels of whole cabinet servers is provided, which detects labels with special features such as QR codes or bar codes on chicken house equipment.

[0047] In another related technology, a label detection method is provided: obtaining appearance description information of an electronic device to be detected; straightening a panel surface picture with a label attached; capturing a label image on the straightened picture according to the label information in the appearance description information; extracting gradient direction histogram features on the captured label image; inputting the extracted gradient direction histogram features into a trained support vector machine (SVM) classifier for classification prediction to obtain a classification prediction result; if the classification prediction result is consistent with the label category recorded in the appearance description information, it is determined that the label of the electronic device to be detected is correctly attached; if the classification prediction result is inconsistent with the label category recorded in the appearance description information, it is determined that the label of the electronic device to be detected is incorrectly attached.

[0048] Most of the device tag detection and positioning identification methods given in the above-mentioned related technologies are based on traditional signal processing or traditional computer vision key point detection methods. These solutions have the following technical disadvantages:

[0049] 1. Insufficient positioning accuracy: The traditional processing method uses a rectangular positioning method, which cannot accurately locate the exact position of the label, especially in the case of complex background or partial occlusion of the label, which seriously affects the reliability of label detection.

[0050] 2. Dependence on camera quality: The detection accuracy of the above methods is highly dependent on the resolution of the camera. Low-resolution cameras will lead to a significant decrease in detection accuracy, limiting the universality and flexibility in practical applications.

[0051] 3. Limited ability to process diverse labels: Traditional processing methods usually rely on specific label features (such as barcodes or QR codes), have poor detection effects on non-standardized labels (such as handwritten labels or name tags), and are difficult to adapt to diverse label types.

[0052] In order to overcome the problems existing in the related art, the embodiments of the present application provide a tag recognition method, device, electronic device, medium and computer program product. The tag recognition method provided in the embodiments of the present application can improve the recognition and extraction of edge features of the tag image, and further obtain more accurate tag information based on the processed edge features.

[0053] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the embodiments provided herein are only used to explain the embodiments of the present application and are not intended to limit the embodiments of the present application. In addition, the embodiments provided below are partial embodiments for implementing the present application, rather than providing all embodiments for implementing the present application. In the absence of conflict, the technical solutions recorded in the embodiments of the present application can be implemented in any combination.

[0054] It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or device. In the absence of further restrictions, an element defined by the sentence "includes..." does not exclude the presence of other related elements (such as steps in the method or units in the device, such as a unit in the device may be a part of a circuit, a part of a processor, a part of a program or software, etc.) in the method or device including the element.

[0055] The tag identification method provided in the embodiment of the present application includes a series of steps, but the tag identification method provided in the embodiment of the present application is not limited to the recorded steps. Similarly, the tag identification device provided in the embodiment of the present application includes a series of modules, but the device provided in the embodiment of the present application is not limited to including the modules explicitly recorded, and may also include modules that need to be set for obtaining relevant information or performing processing based on information.

[0056] The present application embodiment provides a tag identification method, such as Figure 1 As shown, Figure 1 A flow chart of a tag identification method is shown. Figure 1 The tag identification method shown includes:

[0057] Step 101: Process the labeled image to obtain a first feature map corresponding to the image.

[0058] Before tag recognition, an image acquisition device is used to acquire images of the tag. The image acquisition device can acquire images of the tag at different angles, under different lighting conditions, and in different environments. The images of the tag obtained can be images of the tag at different shooting angles or after the tag is rotated by a certain angle.

[0059] After obtaining the labeled image, the labeled image is processed based on a convolutional neural network (CNN) or other neural network to obtain a first feature map corresponding to the image. In an embodiment of the present application, the labeled image can be processed based on an efficient real-time object detection model YOLO (You Only Look Once) to obtain a first feature map corresponding to the image.

[0060] Taking YOLO version (version, v) 5, namely the YOLO v5 model, as an example, YOLO v5 is mainly composed of four parts, namely: input layer, backbone network (Backbone), neck layer and detection head (Head) layer. Among them, the input layer is mainly responsible for data preprocessing and enhancement, the backbone network is mainly responsible for extracting useful feature information from the input image, the neck layer is mainly responsible for fusing and enhancing the features extracted by the backbone network to improve the accuracy of target detection, and the head layer is mainly responsible for outputting information such as the location and category of the image according to the fused feature map.

[0061] Step 102: Obtain a eigenvalue of the first channel dimension according to a global average value of the first channel dimension of the first feature map, wherein the eigenvalue represents the degree of deviation of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map.

[0062] In the process of processing labeled images based on CNN, for color images, the channel dimension of the input layer is usually three, corresponding to the three color channels of red, green, and blue (RGB); for grayscale images, the channel dimension of the input layer can be one. In the convolution layer of CNN, each filter generates a new channel dimension, that is, in the convolution layer, the number of channel dimensions corresponding to the feature map can be consistent with the number of filters in the convolution layer.

[0063] After calculating and obtaining multiple channel dimensions corresponding to the first feature map, any one of the multiple channel dimensions, namely the first channel dimension, is selected, and each value in the first channel dimension is pooled to obtain the global average value of the first channel dimension. Taking formula (1) as an example, the global average value of the first channel dimension is calculated by formula (1).

[0064]

[0065] Among them, c can be any number of the first channel dimension, μ c represents the global average of the first channel dimension, H and W are the height and width of the feature map respectively, x c,i,j Represents the value of the first channel dimension of the feature map X at position (i, j).

[0066] After calculating the global average value of the first channel dimension, the degree of deviation of each value in the first channel dimension relative to the global average value can be obtained based on the global average value of the first channel dimension. For example, based on each value in the first channel dimension and the global average value, any one of the standard deviation, relative deviation, cosine similarity, etc. of each value in the first channel dimension is calculated to obtain the degree of deviation of each value in the first channel dimension relative to the global average value, and the degree of deviation is used as the characteristic value of the first channel dimension.

[0067] Step 103: Determine the weight of the first channel dimension based on the eigenvalue of the first channel dimension; the weight of the first channel dimension is negatively correlated with the eigenvalue of the first channel dimension.

[0068] After obtaining the eigenvalue of the first channel dimension, since the eigenvalue represents the degree of deviation of each value in the first channel dimension relative to the global average value, the larger the eigenvalue, the higher the discreteness of the values ​​in the first channel dimension and the more dispersed the data distribution.

[0069] Since the values ​​in the first channel dimension represent the number of different feature maps in the feature map, the smaller the deviation of the values ​​in the first channel dimension, the more concentrated the values ​​in the first channel dimension are in a certain area or certain pixels, which means that the features represented by the first channel dimension are more significant in the image, that is, the edge features corresponding to the values ​​in the first channel dimension are more significant. Therefore, based on the feature value of the first channel dimension, when the feature value is smaller, that is, the deviation of the value of the first channel dimension is smaller, it means that the edge features corresponding to the first channel dimension are more significant, and the weight of determining the first channel dimension is larger; conversely, when the feature value is larger, that is, the deviation of the value of the first channel dimension is larger, the weight of determining the first channel dimension is smaller.

[0070] Step 104: Based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension, a processed feature map is obtained; and according to the processed feature map, label information is output.

[0071] Based on the above steps, the weight of each channel dimension in the first feature map is obtained, that is, after the weights of multiple first channel dimensions are obtained, the multiple first channel dimensions are weighted to obtain a processed feature map. For example, the processed feature map is calculated by the YOLO v5 model, and label information is obtained based on the processed feature map, wherein the label information calculated and output based on the YOLO v5 model includes the x, y coordinate points, width, height, rotation angle of the detection frame, the probability of an object in the detection frame, and the object category of the detection frame.

[0072] The embodiment of the present application provides a label recognition method, which determines the weight of the first channel dimension by the discrete degree of each value in the first channel dimension, and obtains a processed feature map by weighted processing of multiple first channel dimensions, which can further highlight the edge features of the label image. Therefore, the accuracy of the label information obtained through the processed feature map is higher.

[0073] In practical applications, steps 101 to 104 can be implemented based on a processor, and the processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0074] In some embodiments, the feature value is the variance corresponding to each value in the first channel dimension.

[0075] The degree of deviation of each value in the first channel dimension relative to the global average value in step 102 can be obtained by calculating the variance of each value in the first channel dimension and the global average value. The variance corresponding to each value in the first channel dimension and the global average value can be obtained based on formula (2).

[0076]

[0077] in, Represents the variance of each value of the first channel dimension relative to the global average, μ c represents the global average of the first channel dimension, μ cIt can be obtained by formula (1), H and W are the height and width of the feature map respectively, x c,i,j Represents the value of the first channel dimension of the feature map X at position (i, j).

[0078] In some embodiments, the above-mentioned processing of the image of the label to obtain the first feature map corresponding to the image includes: processing the image of the label to obtain a second feature map; calculating the first value in the second feature map to obtain the first feature map corresponding to the image; wherein the first value is each value in each channel of the second feature map; the first feature map includes multiple second values, and the second values ​​correspond to the first values ​​one by one; when the first value is greater than 0, the second value is greater than or equal to the first value; when the first value is less than 0, the second value is less than or equal to 0.

[0079] Taking the YOLO v5 model as an example, the labeled image is processed. First, the labeled image is processed based on the attention Focus module in the backbone network of the YOLO v5 model to obtain the second feature map. The second feature map corresponds to multiple channel dimensions, wherein each of the multiple channel dimensions corresponding to the second feature map includes multiple first values, and these first values ​​can represent the feature intensity of a specific area in the image. Specifically, the information extracted from the labeled image by convolution calculation, that is, the first value represents the edge feature of the image. In addition to the edge feature, the first value can also characterize the texture, shape and other information of the image. The larger the first value, the greater the possibility that the pixel is an edge; conversely, the smaller the value, the smaller the possibility that the pixel is an edge.

[0080] This implementation provides a method for processing a first value based on the edge characteristics characterized by the first value. After obtaining the second feature map, each first value in each of the multiple channels corresponding to the second feature map is processed to obtain a second value, so that the second value corresponding to the first value is equal to 0 or less than 0 when it is less than 0, so as to reduce the influence of the first value less than 0 on the subsequent feature map calculation, that is, to increase the influence of the first value greater than 0 on the feature map, and increase the edge characteristics of the feature map.

[0081] In some embodiments, the first value may be further processed. When the first value is greater than 0, the processed first value, i.e., the second value, is made close to the first value. When the first value approaches negative infinity from 0, the second value is made to approach 0 from -1 accordingly.

[0082] By processing the first value and changing the first value, the edge features of the image mapped by the first value in the second feature map can be "enlarged", which is further beneficial to obtain more significant edge features through the first feature map composed of the second values ​​and output accurate label information.

[0083] In some embodiments, the above-mentioned calculation of the first value in the second characteristic map includes: calculating the first value at least by using a hyperbolic tangent function.

[0084] Based on the distribution characteristics of the hyperbolic tangent function, the first value is used as the dependent variable of the hyperbolic tangent function, and the independent variable of the hyperbolic tangent function is further processed, which is conducive to achieving the effect of amplifying edge features.

[0085] In some embodiments, the argument of the hyperbolic tangent function is positively correlated with the first value.

[0086] By setting the independent variable of the hyperbolic tangent function to be positively correlated with the first value, compared with directly using the first value as the dependent variable of the hyperbolic tangent function, the second value can be obtained based on the first value, thereby effectively extracting the edge features of the image.

[0087] In some embodiments, the first value may be calculated based on the function in formula (3) to obtain the second value, wherein x in formula (3) represents the first value and f(x) represents the second value.

[0088] f(x) = x·tanh (ln (1+e x )) (3)

[0089] The image of the function corresponding to formula (3) is as follows Figure 2 As shown, it can be seen that when the first value is greater than 0, the size of the corresponding second value calculated by the first value is basically equal to the first value. When the first value is less than 0 and greater than -5, the value of the corresponding second value calculated by the first value is less than 0 and greater than the first value; when the first value is less than -5, the value of the corresponding second value calculated by the first value is infinitely close to 0.

[0090] Based on formula (3), each first value in each channel of the second feature map is calculated to obtain the first feature map. Compared with the second feature map before processing, the first feature map can better reflect the edge features of the image. Further processing of the first feature map is conducive to extracting more effective edge features from the image, so that the label recognition model has higher accuracy and generalization.

[0091] In addition, according to the numerical characteristics of each channel dimension in the feature map, the part with smaller values ​​often corresponds to the low-frequency image, that is, the corresponding pixel change smooth area, while the high-frequency image often shows a drastic change in value in the feature map. Therefore, based on formula (3), the first feature map is obtained by calculating each value in each channel dimension of the second feature map, which can not only retain the information of the low-frequency image, but also retain the information of the high-frequency image.

[0092] Figure 3A schematic diagram of the model structure after improving the YOLO v5 model based on the label recognition method given in the above embodiment is shown.

[0093] according to Figure 3 As shown, the input layer (not shown) preprocesses the labeled image and then inputs the preprocessed image into the backbone network. The Focus module of the backbone network compresses and integrates the preprocessed image, extracts higher-level feature representations, and outputs a second feature map. The device label convolution module in the backbone network processes the second feature map output by the Focus module, as shown in FIG. Figure 4 As shown in FIG. 1 , the newly added device label convolution module in the backbone network can be specifically composed of three parts: a convolution submodule, a batch normalization (BN) submodule, and an activation submodule, wherein the activation submodule includes the calculation logic given by the above formula (3), further processes the first value output by the BN submodule, and inputs the processed first value to the Neck layer, and each module in the Neck further processes the first value to obtain the first feature map input to the Head layer, and each second value in each channel dimension of the first feature map.

[0094] in addition, Figure 3 The backbone network shown also includes a cross-stage partial network (CSP), which is used to divide the input features into two parts and then cross-connect the two parts to improve the performance of the deep neural network, where CSP1-1, CSP1-2, and CSP1-3 represent different cross-stage partial networks for processing the data output by the convolution modules of different device labels. The spatial pyramid pooling (SPP) in the backbone network is used to convert feature maps of any size into feature vectors of fixed size for subsequent tasks such as feature classification or regression. Figure 3 The CSP2-1 in the Neck layer shown represents a cross-stage local network in the Neck layer, and the concatenation CONCAT in the Neck layer is used to concatenate two or more feature maps in a specific dimension. The convolution Mish and convolution ReLU in the Neck layer represent different activation functions. The Conv in the Head layer represents a convolution layer, which is used to continue to perform convolution operations on the feature map to extract higher-level feature information.

[0095] In order to improve the algorithm's ability to distinguish between labels and backgrounds, the embodiment of the present application also introduces an attention mechanism in the Head layer to enhance the ability to represent label features. Through the method provided in the embodiment of the present application, the attention module in the related art can be replaced by the newly added label edge feature attention module in the YOLO v5 model, and the weight of each channel dimension is calculated by the label edge feature attention module, and different weights are assigned to each channel dimension.

[0096] Figure 5 The processing flow diagram of the label edge feature attention module is shown. The label edge feature attention module can calculate the feature map, such as pooling each value in the first channel dimension, to obtain the attention weight, and then adjust the weights of each first channel dimension of the input first feature map, which is conducive to suppressing unimportant features in the first feature map. In order to explore the influence of each first channel dimension on the model without adding too many parameters, the input feature map is set to: X∈R C×H×W , where R C×H×W is a set of real numbers, representing a three-dimensional tensor corresponding to C×H×W, where C is the number of channels, H and W are the height and width of the feature map, respectively. The processing method of the label edge feature attention module includes:

[0097] Step 501: Input a first feature map.

[0098] The first feature map obtained after being processed by the backbone network and the Neck layer is input into the label edge feature attention module, wherein the input first feature map includes each value of the first channel dimension of the first feature map.

[0099] Step 502: Pooling the first feature map.

[0100] The global average value of the first channel dimension can be calculated by the above formula (1).

[0101] Step 503: Calculate the eigenvalue of the channel dimension.

[0102] The variance of the first channel dimension can be calculated by the above formula (2), and the calculated variance is used as the eigenvalue of the first channel dimension.

[0103] Step 504: Calculate weights.

[0104] Based on the method given in the above steps, the weight of the first channel dimension can be calculated based on the variance of the first channel dimension. For example, the weight of the first channel dimension can be calculated by formula (4).

[0105]

[0106] Among them, β crepresents the weight of the first channel dimension, μ c represents the global average of the first channel dimension, represents the variance of the first channel dimension, ∈ is a small positive number preset based on experience to prevent the denominator of formula (4) from being zero.

[0107] Step 505: Adjust the first feature map.

[0108] Based on the above formula (4), based on the weight α c Adjust each first channel dimension in the first feature map, that is, α c Applied to the corresponding first channel dimension of the input first feature map X to obtain the processed feature map.

[0109] For example, the processed feature map can be obtained based on formula (5):

[0110] X' c,i,j =α c ·X c,i,j (5)

[0111] Among them, X' c,i,j Represents the value of the processed first feature map at the first channel dimension (i, j), X c,i,j Represents the value of the first feature map X of the input at the position (i, j) of the first channel dimension.

[0112] Step 506: Output the processed feature map.

[0113] After the first feature map processed by the label edge feature attention module is output, it is further processed by the Head layer to output the label information, such as Figure 3 As shown in the figure, the final output label information of the YOLO v5 model includes x, y, w, h, ori, obj, cls, where x and y represent the coordinate points of the detection box in the xy coordinate system, that is, the center point or the upper left corner of the detection box, w represents the width of the detection box, h represents the height of the detection box, ori represents the rotation angle of the label image, obj represents the probability of an object in the detection box, and cls represents the object category in the detection box.

[0114] Based on the tag recognition method given in the above embodiment, a pseudo code implementation is given below:

[0115] 1. import numpy as np

[0116] 2.def label_attn_module(input_feature_map,epsilon=1e-5):

[0117] 3.C,H,W=input_feature_map.shape

[0118] 4.# Compute device label feature map pooling

[0119] 5.mu=np.mean(input_feature_map,axis=(1,2),keepdims=True)

[0120] 6.#Calculate the eigenvalues ​​of the device tag feature map

[0121] 7.sigma_sq=np.mean((input_feature_map-mu)**2, axis=(1,2),keepdims=True)

[0122] 8.#Calculate the weight value of the label edge feature attention module

[0123] 9.alpha=mu**2 / (sigma_sq+epsilon)

[0124] 10.#Adjust device label feature map

[0125] 11.output_feature_map=alpha*input_feature_map

[0126] 12.return output_feature_map

[0127] 13.# Example input feature map

[0128] 14.input_feature_map=np.random.randn(3,32,32)

[0129] 15.#Apply Label Edge Feature Attention Module

[0130] 16.output_feature_map=label_attn_module(input_feature_map)

[0131] The above embodiment provides a method for performing label recognition based on a label recognition model. The following provides a method for performing model training on the label recognition model.

[0132] In some embodiments, before processing the labeled image, the above method also includes: performing data enhancement on the training image, performing model training based on the data enhanced training image, and obtaining a label recognition model; the label recognition model is used to process the labeled image and output the label information; wherein, performing data enhancement on the training image includes: increasing the shadow of the training image, and / or increasing the brightness of the training image.

[0133] In the process of label recognition training of the model, various transformations are applied to the existing image data to generate more training samples. More training samples can help the model generalize better and thus perform more stably in actual scenarios.

[0134] This embodiment provides a method of adding shadow processing to the training image and / or enhancing the brightness of the training image when training the model. In the actual training process, the training image can be processed, for example, by adjusting the color scale of the image, or adding different lighting processing to the collected object during image acquisition.

[0135] By increasing the shadow of the training image and / or improving the brightness of the training image, a data-enhanced training image can be obtained, and then the model is trained with the data-enhanced training image to obtain a label recognition model with stronger recognition ability. The label recognition model obtained after training here is used to identify the label based on the method given in the above embodiment and output the label information.

[0136] In some embodiments, the above-mentioned increase in the shadow of the training image includes: multiplying the mask function by the pixel value of the training image to increase the shadow of the processed training image; the value range of the mask function is greater than 0 and less than 1; the mask function is positively correlated with the pixel value before modification.

[0137] To increase the shadow of the training image, the pixel value of the original training image may be multiplied by a mask function with a value range between 0 and 1 on the basis of the original training image, so as to reduce the pixel value of the original training image and increase the shadow of the original training image.

[0138] For example, the shadow of the training image can be added by calculating formula (6).

[0139] I shadow (x,y)=I(x,y)·M(x,y) (6)

[0140] Among them, M(x,y) is a mask function whose value varies between 0 and 1 and is used to simulate the shadow intensity on the label; I(x,y) represents the pixel value.

[0141] For example, the mask function M(x,y) can be obtained based on formula (7):

[0142]

[0143] Further combining formula (6) and formula (7), we obtain the calculation method for adding shadows to the training image based on formula (8):

[0144]

[0145] In some embodiments, the above-mentioned improving the brightness of the training image includes: adding a brightening function to the pixel value of the training image to improve the brightness of the processed training image; the brightening function is negatively correlated with the pixel value before modification.

[0146] It can be seen that by increasing the pixel value of the training image, the brightness of the training image can be improved. Furthermore, when the pixel value of the training image is high, it means that the original brightness of the training image is high, so a smaller brightening function is added to the pixel value of the training image with a higher pixel value; conversely, when the pixel value of the training image is low, it means that the original brightness of the training image is low, so a larger brightening function is added to the pixel value of the training image with a lower pixel value to improve the brightness of the training image.

[0147] For example, the brightness of the training image can be increased by using formula (9).

[0148] I glare (x,y)=I(x,y)+G(x,y) (9)

[0149] Among them, G(x,y) is a brightening function used to increase the brightness of the training image.

[0150] For example, the brightening function G(x,y) can be obtained based on formula (10):

[0151]

[0152] Further combining formula (9) and formula (10), we obtain the calculation method for improving the brightness of the training image based on formula (11):

[0153]

[0154] By using the method of increasing the shadow of the training image and improving the brightness of the training image provided in the above embodiment, data enhancement is performed on the training image, which can effectively simulate different lighting conditions and make the model more robust to various lighting changes in practical applications, thereby improving its performance and reliability in different environments.

[0155] Figure 6 and Figure 7 The training image set used for model training is shown as Figure 6 and Figure 7 As shown, the training images can be divided into five categories of labels, including: 'QRCode' two-dimensional code, 'label' equipment label, 'nameplate' equipment nameplate (metal), 'serial_no' serial number, 'inkjet_code' inkjet code, etc.

[0156] In some embodiments, the processing of the training image may further include: rotating, flipping, translating, scaling, adjusting brightness, adjusting contrast, and the like.

[0157] Specifically, the rotation process refers to rotating the training image at a random angle to increase the number of training images with different viewing angles. The flip process refers to flipping the training image horizontally or vertically to simulate different views. The translation process refers to translating the training image horizontally or vertically to change the position of the object in the training image. The scaling process refers to scaling the training image to change the size of the object.

[0158] The brightness adjustment process can be performed on the pixel values ​​of the training image based on formula (12).

[0159] I brightness (x,y)=I(x,y)+ΔB (12)

[0160] Among them, ΔB is a random constant that represents the change in brightness.

[0161] The contrast adjustment process can be performed on the pixel values ​​of the training image based on formula (13).

[0162] I contrast (x,y)=q·(I(x,y)-μ)+μ (13)

[0163] Among them, q is a random coefficient, which indicates the change of contrast; μ indicates the average pixel value of the training image:

[0164]

[0165] The label recognition model obtained based on the training method given in this implementation recognizes labels, and the recognition results are significantly improved compared with the existing YOLOv5 Oriented Bounding Boxes (OBB) model. For specific comparison data, see Table 1 below, where F1 represents the balanced F score, which is an indicator used to measure the accuracy of the binary classification model.

[0166] Table 1

[0167]

[0168] Based on the label recognition method given in the embodiment of the present application, by adding the device label convolution module and the label edge feature attention module, the edge features in the feature map of the label image can be improved, and the accuracy of the output label information can be improved. By increasing the shadow of the training image, improving the brightness of the training image, etc., the robustness of the label recognition model in different shooting situations can be enhanced.

[0169] The tag recognition method provided in the embodiment of the present application can support the recognition of tag characteristics such as multiple categories and multiple angles; the device tag recognition method using a rotation angle has better detection effect and higher accuracy than the related technology, and can predict the rotation angle of the tag.

[0170] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0171] Based on the tag recognition method proposed in the above embodiment, the present application embodiment also provides a tag recognition device, such as Figure 8 As shown, the label identification device includes:

[0172] The first processing module 801 is used to process the label image to obtain a first feature map corresponding to the image.

[0173] The second processing module 802 is used to obtain the eigenvalue of the first channel dimension according to the global average value of the first channel dimension of the first feature map, wherein the eigenvalue represents the degree of deviation of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map; based on the eigenvalue of the first channel dimension, determine the weight of the first channel dimension; the weight of the first channel dimension is negatively correlated with the eigenvalue of the first channel dimension.

[0174] Identification module 803: used to obtain a processed feature map based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension; output label information according to the processed feature map.

[0175] In practical applications, the first processing module 801, the second processing module 802, and the identification module 803 can be implemented based on a processor and a communication device.

[0176] In some embodiments, the first processing module 801 is specifically used to process the image of the label to obtain a second feature map; calculate the first value in the second feature map to obtain the first feature map corresponding to the image; wherein the first value is each value in each channel of the second feature map; the first feature map includes multiple second values, and the second values ​​correspond to the first values ​​one by one; when the first value is greater than 0, the second value is greater than or equal to the first value; when the first value is less than 0, the second value is less than or equal to 0.

[0177] In some embodiments, the first processing module 801 is specifically configured to calculate the first value at least by using a hyperbolic tangent function.

[0178] In some embodiments, the label recognition device also includes a training module, which is used to perform data enhancement on the training image, and perform model training based on the training image after data enhancement to obtain a label recognition model; the label recognition model is used to process the label image and output the label information; wherein the training module is specifically used to increase the shadow of the training image and / or enhance the brightness of the training image.

[0179] In some embodiments, the training module is specifically used to multiply the mask function by the pixel value of the training image to increase the shadow of the processed training image; the value range of the mask function is greater than 0 and less than 1; the mask function is positively correlated with the pixel value before modification.

[0180] In some embodiments, the training module is specifically used to add a brightening function to a pixel value of a training image to improve the brightness of the processed training image; the brightening function is negatively correlated with the pixel value before modification.

[0181] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the same method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0182] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0183] An embodiment of the present application also provides an electronic device. Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the electronic device 90 may include:

[0184] The memory 901 is used to store executable instructions.

[0185] The processor 902 is configured to implement any one of the above-mentioned tag identification methods when executing the executable instructions stored in the memory 901.

[0186] The processor 902 may be at least one of an ASIC, a DSP, a DSPD, a PLD, a FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

[0187] The above-mentioned computer-readable storage medium or memory 901 can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and other memories; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0188] An embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored, and the computer executable instructions are used to implement any one of the tag identification methods provided in the above embodiments.

[0189] Correspondingly, an embodiment of the present application further provides a computer program product, which includes computer executable instructions, and the computer executable instructions are used to implement any one of the tag identification methods provided in the above embodiments.

[0190] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0191] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0192] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0193] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0194] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0195] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0196] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A tag recognition method, characterized in that: The method comprises: Processing the labeled image to obtain a first feature map corresponding to the image; Obtaining a feature value of the first channel dimension according to a global average value of the first channel dimension of the first feature map, wherein the feature value represents a degree of deviation of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map; Determining a weight of the first channel dimension based on the eigenvalue of the first channel dimension; the weight of the first channel dimension is negatively correlated with the eigenvalue of the first channel dimension; Based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension, a processed feature map is obtained; and according to the processed feature map, information of the label is output.

2. The method according to claim 1, characterized in that The eigenvalue is the variance corresponding to each value in the first channel dimension.

3. The method according to claim 1 or 2, characterized in that: The step of processing the labeled image to obtain a first feature map corresponding to the image includes: Processing the labeled image to obtain a second feature map; Calculate the first value in the second feature map to obtain the first feature map corresponding to the image; wherein the first value is each value in each channel of the second feature map; the first feature map includes multiple second values, and the second values ​​correspond to the first values ​​one by one; when the first value is greater than 0, the second value is greater than or equal to the first value; when the first value is less than 0, the second value is less than or equal to 0.

4. The method according to claim 3, characterized in that: The calculating the first value in the second characteristic graph includes: The first value is calculated at least by a hyperbolic tangent function.

5. The method according to claim 4, characterized in that The independent variable of the hyperbolic tangent function is positively correlated with the first value.

6. The method according to claim 1, characterized in that Before processing the label image, the method further includes: Performing data enhancement on the training images, and performing model training based on the data enhanced training images to obtain a label recognition model; the label recognition model is used to process the labeled images and output label information; wherein, performing data enhancement on the training images includes: The shadow of the training image is increased, and / or the brightness of the training image is enhanced.

7. The method according to claim 6, characterized in that The step of adding a shadow to the training image comprises: The mask function is multiplied by the pixel value of the training image to increase the shadow of the processed training image; the value range of the mask function is greater than 0 and less than 1; and the mask function is positively correlated with the pixel value before modification.

8. The method according to claim 6, characterized in that The step of improving the brightness of the training image comprises: A brightening function is added to the pixel value of the training image to improve the brightness of the processed training image; the brightening function is negatively correlated with the pixel value before modification.

9. A label recognition device, characterized in that: The device comprises: A first processing module, used to process the label image to obtain a first feature map corresponding to the image; A second processing module is configured to obtain a feature value of the first channel dimension according to a global average value of the first channel dimension of the first feature map, wherein the feature value represents a degree of deviation of each value in the first channel dimension relative to the global average value; the first channel dimension is any channel dimension of the first feature map; based on the feature value of the first channel dimension, a weight of the first channel dimension is determined; and the weight of the first channel dimension is negatively correlated with the feature value of the first channel dimension; An output module is used to obtain a processed feature map based on the value of the first feature map in the first channel dimension and the weight of the first channel dimension; and output the label information according to the processed feature map.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory for storing a computer program that can be run on the processor; wherein, The processor is configured to run the computer program to perform the method according to any one of claims 1 to 8.

11. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1 to 8 when executed by a processor.

Images