Method for identifying internal production date of bottle cap and detection identifier

Abstract

The present application relates to the technical field of image processing, and specifically refers to a bottle cap internal production date recognition method and a detection recognizer, which comprises: inputting a preprocessed bottle cap image into a character detection model, using an adaptive feature fusion module to perform feature fusion on two feature maps of adjacent scales respectively, obtaining a third fusion feature map of the two feature maps of adjacent scales, and finally outputting four vertex coordinates of each row of character boxes in the bottle cap image; according to the four vertex coordinates of each row of character boxes in the bottle cap image, using an angle adjuster to judge the position of the character box, and then using a perspective transformation algorithm to cut out each row of character region images in the bottle cap image in combination with a preset cutting size; inputting each row of character region images in the bottle cap image into a character recognition model to output each row of text sequences corresponding to the bottle cap image; wherein the first row of text sequences corresponding to the bottle cap image is the recognized bottle cap internal production date. The present application improves the accuracy of recognizing the production date of the bottle cap at any angle.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and detector for identifying the production date inside a bottle cap. Background Technology

[0002] In the food and beverage industry, the production date inside the bottle cap is crucial information for product traceability and quality control. Traditional methods for detecting production dates mainly rely on manual sampling. For large-scale production, manual inspection is not only time-consuming and labor-intensive, but also inefficient, prone to errors such as missed or false detections, severely impacting production efficiency, product reputation, and consumer trust.

[0003] In recent years, with the development of computer vision and deep learning technologies, image recognition-based methods have gradually become mainstream. Although these technologies have greatly improved the efficiency and accuracy of detection through automation, existing optical character recognition (OCR) technologies do not perform well when processing bottle cap images from any angle, especially in images of bottle caps moving rapidly on the production line, where the text may be at various tilt angles, increasing the difficulty of recognition. This makes it difficult to accurately identify the production date inside the bottle cap, making it difficult to accurately trace problematic product batches and isolate defective products in a timely manner, thereby increasing operating costs and reducing production efficiency and resource utilization. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the low accuracy of the existing technology in recognizing bottle cap images from arbitrary angles and moving rapidly, which makes it difficult to accurately trace problematic product batches, and unable to isolate defective products in a timely manner, thereby increasing operating costs and reducing production efficiency and resource utilization.

[0005] To solve the above-mentioned technical problems, the present invention provides a method for identifying the production date inside a bottle cap, comprising:

[0006] Obtain the bottle cap image, preprocess the bottle cap image, and obtain the preprocessed bottle cap image;

[0007] The preprocessed bottle cap image is input into the backbone network of the text detection model, and the output is the feature map of the bottle cap image at various scales.

[0008] The feature maps at various scales of the bottle cap image are input into the neck network of the text detection module. An adaptive feature fusion module is then used to fuse the features of two feature maps at adjacent scales, outputting a third fused feature map of the two adjacent scales, including:

[0009] The bottle cap image is fed into the adjacent low-scale and high-scale feature maps to the initial feature fusion unit, which outputs the first fused feature map of the two adjacent feature maps. The first fused feature map is then fed into the channel attention unit, which outputs the weights corresponding to the low-scale feature map and the high-scale feature map. The low-scale feature map and its corresponding weights, along with the high-scale feature map and its corresponding weights, are fed into the feature fusion unit again, which outputs the second fused feature map of the two adjacent feature maps. The second fused feature map is then fed into the spatial attention unit, which outputs the third fused feature map of the two adjacent feature maps.

[0010] The third fused feature map of all two adjacent feature maps is input into the output layer of the neck network to output the final fused feature map of the bottle cap image.

[0011] The final fused feature map of the bottle cap image is input into the head network of the text detection model, and the predicted probability map corresponding to the bottle cap image is output. Based on the predicted probability map corresponding to the bottle cap image, the coordinates of the four vertices of the first row of text boxes and the coordinates of the four vertices of the second row of text boxes in the bottle cap image are obtained.

[0012] Using an angle adjuster, in a clockwise direction from left to right, the coordinates of the four vertices of the first line of text boxes in the bottle cap image are sequentially recorded as the first coordinate point, the second coordinate point, the third coordinate point, and the fourth coordinate point. If the distance between the first and second coordinate points is less than or equal to the distance between the second and third coordinate points, then based on the vertex coordinates of the first line of text boxes in the bottle cap image, a perspective transformation algorithm is used, combined with a preset cutting size, to cut out the first line of text area image in the bottle cap image. If the distance between the first and second coordinate points is greater than the distance between the second and third coordinate points, then the first and third coordinate points are swapped. After obtaining the new coordinates of the four vertices of the first line of text boxes in the bottle cap image, a perspective transformation is used, combined with a preset cutting size, to cut out the first line of text area image in the bottle cap image.

[0013] Using the angle adjuster, the text area of ​​the second row in the bottle cap image is cut out based on the coordinates of the four vertices of the text box in the second row in the bottle cap image;

[0014] Input the first and second line text regions from the bottle cap image into the character recognition model, and output the first and second line text sequences corresponding to the bottle cap image; wherein, the first line text sequence corresponding to the bottle cap image is the recognized production date inside the bottle cap.

[0015] Preferably, the backbone network of the text detection model is a lightweight residual network ResNet18, which includes four residual blocks; the number of channels in the four residual blocks are 64, 128, 256, and 512, respectively.

[0016] Preferably, the step of inputting the adjacent low-scale feature maps and high-scale feature maps of the bottle cap image into the initial feature fusion unit, and outputting the first fused feature map of the two feature maps at the current adjacent scales, is expressed as follows:

[0017] ;

[0018] in, This represents the first fused feature map of two feature maps at adjacent scales. This represents the low-scale feature map among two adjacent feature maps corresponding to the bottle cap image; This represents the high-scale feature map corresponding to two adjacent feature maps of the bottle cap image.

[0019] Preferably, the step of inputting the low-scale feature map and its corresponding weights and the high-scale feature map and its corresponding weights into the feature re-fusion unit, and outputting the second fused feature map of the two feature maps at the current adjacent scales, is expressed as follows:

[0020] ;

[0021] in, This represents the second fusion feature map of the bottle cap image; This represents the low-scale feature map among two adjacent feature maps corresponding to the bottle cap image; This represents the high-scale feature map corresponding to two adjacent feature maps of the bottle cap image; This indicates the weights of the lower-scale feature maps in two adjacent feature maps at the current scale. This represents the weights of the higher-scale feature maps in the two adjacent feature maps.

[0022] Preferably, it further includes:

[0023] Acquire multiple bottle cap images and construct an image dataset;

[0024] Using an image annotation tool, the first and second lines of text in each bottle cap image in the image dataset are annotated to obtain the coordinates of the four vertices of the first and second line text boxes in each bottle cap image, and a coordinate information label set is constructed; based on the image dataset and its corresponding coordinate information label set, a text detection dataset is constructed.

[0025] The text detection dataset is preprocessed by adjusting all bottle cap images in the corresponding image dataset to 1280*720 pixels, and scaling all vertex coordinates in the coordinate information label set proportionally to obtain the preprocessed text detection dataset.

[0026] Based on the coordinate information labels of each bottle cap image in the preprocessed text detection dataset, for any pixel in each bottle cap image, if the current pixel is inside the text box, the probability value of the current pixel is 1; if the current pixel is outside the text box, the probability value of the current pixel is 0. This generates a true probability map for each bottle cap image. The probability value of each pixel is used as the true label of each pixel.

[0027] Scaling and dilution operations are performed on the true probability map of each bottle cap image to obtain the true threshold map of each bottle cap image, the expression of which is:

[0028] ;

[0029] in, Indicates the first True threshold map of bottle cap image; Indicates the first Area of ​​the true probability graph of the bottle cap image; Indicates the coefficient of shrinkage; Indicates the first The perimeter of the true probability map of the bottle cap image; the value of each pixel in each true threshold map is the threshold value for each pixel;

[0030] Using a differentiable binarization method, based on the true probability map and true threshold map of each bottle cap image, a true approximate binarized map of each bottle cap image is obtained, and its expression is:

[0031] ;

[0032] in, Indicates the first The true approximate binary image of the bottle cap image is in coordinates. Pixel value at; Indicates the first The true probability graph of the bottle cap image is shown on the coordinates. Pixel value at; Indicates the first The true threshold map of the bottle cap image is shown in the coordinates. Pixel value at; Represents a constant; if Then the true approximate binary map is in coordinates The pixel at this location belongs to the text box, and its coordinates are... The pixel at that location corresponds to the real label 1; if Then the true approximate binary map is in coordinates The pixel at this location is outside the text box; at this point, its coordinates are... The actual label corresponding to the pixel at that location is 0;

[0033] Based on the true probability map, true threshold map, and true approximate binary map of each bottle cap image, a set of true image labels is constructed.

[0034] Based on the preprocessed text detection dataset and the set of real image labels, a text detection data label set is constructed;

[0035] The text detection data label set is divided into a 70% text detection training set and a 30% text detection test set;

[0036] To minimize the difference between the predicted probability and the true label of each pixel, a probabilistic graphical loss function is constructed; to minimize the error between the predicted threshold map and the true threshold map, a threshold map loss function is constructed; to minimize the difference between the predicted approximate binary map and the true approximate binary map, an approximate binary map loss function is constructed; and based on the probabilistic graphical loss function, the threshold map loss function, and the approximate binary map loss function, a text detection loss function is constructed.

[0037] Using the text detection data label set, the initial text detection model is trained by minimizing the text detection loss function, and the trained initial text detection model is used as the text detection model.

[0038] Cut the image according to the preset cutting size and the actual coordinates of the four vertices of each line of text boxes in each bottle cap image to obtain the first line of text region image and the second line of text region image in each bottle cap image.

[0039] Using image annotation tools, the real text data and path source of the first and second lines of text information in each bottle cap image in the image dataset are annotated, and a real text label set is constructed; based on the text region images corresponding to each line of text in each bottle cap image in the image dataset and their corresponding real text label sets, a text recognition dataset is constructed.

[0040] The text recognition dataset is preprocessed by adjusting the text region images corresponding to each line of text in each bottle cap image to 180*32 pixels to obtain the preprocessed text recognition dataset.

[0041] The preprocessed text recognition dataset was divided into a 70% text recognition training set and a 30% text recognition test set.

[0042] The connection-time classification loss function is used as the character recognition loss function; using the character recognition training set, the initial character recognition model is trained by minimizing the character recognition loss function, and the trained initial character recognition model is used as the character recognition model.

[0043] Preferably, the expression for the text detection loss function is:

[0044] ;

[0045] in, This represents the loss function of the text detection model; This represents the probabilistic graphical loss function; This represents the threshold plot loss function; This represents the loss function for an approximate binary graph. Indicates the threshold graph weights; Indicates the weights of the binary graph;

[0046] The expression for the probabilistic graphical loss function is:

[0047] ;

[0048] in, This represents the set of all pixels in the probability graph; In the probability graph of prediction, the first... The probability value of a pixel being located in a text region; Represents the first in the true probability graph The true label of the nth pixel, if the nth pixel in the true probability map If a pixel is located within the text area, then ,otherwise ;

[0049] The expression for the approximate binary graph loss function is:

[0050] ;

[0051] in, This represents the set of all pixels in an approximate binary image. Indicates the prediction of the first value in the approximate binary graph. The probability value of a pixel being located in a text region; Represents the first in a true approximation binary graph. The true label of the nth pixel, if the true label is approximated by the nth pixel in the binary image If a pixel is located within the text area, then ,otherwise ;

[0052] The expression for the threshold map loss function is:

[0053] ;

[0054] Specifically, a threshold map is obtained by dilating the probability map. This represents the set of all pixels in the threshold image; The first one represents the prediction threshold map. The threshold for each pixel; Represents the true threshold in the graph. The true threshold for each pixel.

[0055] Preferably, the expression for the character recognition loss function is:

[0056] ;

[0057] in, An image representing the text region input to the character recognition model; This represents the actual character sequence corresponding to the text region image in the bottle cap image; This represents the text recognition training set; Represents probability; This represents the set of all possible paths to predict the true character sequence from the input text region image; This represents all possible paths to predict the true character sequence from the input text region image.

[0058] Preferably, the character recognition model adopts a CRNN architecture, including: convolutional layers, recurrent layers, and transcription layers;

[0059] The convolutional layer is composed of six convolutional kernels of size 3 and one convolutional kernel of size 2 stacked together. It is used to extract the features corresponding to the first row of text regions and the second row of text regions in the bottle cap image, respectively, to obtain the feature map corresponding to the first row of text regions and the feature map corresponding to the second row of text regions in the bottle cap image.

[0060] The recurrent layer is a bidirectional long short-term memory network, which is used to perform sequence modeling on the feature maps corresponding to the first row of text regions and the second row of text regions in the bottle cap image, respectively, to obtain the feature sequences corresponding to the first row of text regions and the second row of text regions in the bottle cap image.

[0061] The transcription layer is used to map the feature sequences corresponding to the first and second lines of text regions in the bottle cap image to a preset character set, respectively, to generate the first and second line text sequences corresponding to the bottle cap image; wherein, the first line text sequence corresponding to the bottle cap image is the identified production date inside the bottle cap; and the second line text sequence corresponding to the bottle cap image is the identified time information inside the bottle cap.

[0062] This invention also provides a device for detecting and identifying the production date inside a bottle cap, which applies the aforementioned method for identifying the production date inside a bottle cap, including:

[0063] Image acquisition module: used to acquire bottle cap images and preprocess the bottle cap images to obtain preprocessed bottle cap images;

[0064] Text detection module: Used to obtain the coordinates of the four vertices of the first row of text boxes and the four vertices of the second row of text boxes in the preprocessed bottle cap image using a text detection model;

[0065] The rotation correction module uses an angle adjuster to sequentially assign the coordinates of the four vertices of the first line of text boxes in the bottle cap image to the first, second, third, and fourth coordinate points in a clockwise direction from left to right. If the distance between the first and second coordinate points is less than or equal to the distance between the second and third coordinate points, the first line of text area in the bottle cap image is cut out based on the vertex coordinates of the first line of text boxes using a perspective transformation algorithm and a preset cutting size. If the distance between the first and second coordinate points is greater than the distance between the second and third coordinate points, the first and third coordinate points are swapped to obtain new coordinates for the four vertices of the first line of text boxes in the bottle cap image. Then, perspective transformation is used, combined with a preset cutting size, to cut out the first line of text area in the bottle cap image. Finally, using the angle adjuster, the second line of text area in the bottle cap image is cut out based on the coordinates of the four vertices of the second line of text boxes.

[0066] Text recognition module: Used to obtain the first and second line text sequences corresponding to the bottle cap image based on the first and second line text regions in the bottle cap image; wherein, the first line text sequence in the bottle cap image is the identified production date inside the bottle cap.

[0067] Preferably, it also includes a recording and alarm module, used to match the first line of text sequence in the bottle cap image with the date of production and the second line of text sequence with the time of production. If there is a mismatch in any line of text information, it will be recorded in time and an alarm will be sounded.

[0068] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0069] The present invention discloses a method for identifying the production date inside a bottle cap. This method incorporates an adaptive feature fusion module into the text detection model. Based on the pairwise adjacent low-scale and high-scale feature maps corresponding to the bottle cap image, a final fused feature map of the bottle cap image is obtained through an initial feature fusion unit, a channel attention unit, a secondary feature fusion unit, and a spatial attention unit. These fused features contain both high-level semantic features and low-level detail features, thereby improving the accuracy of the model's detection. Furthermore, perspective transformation is used to convert text regions at any angle into horizontal text regions. Using these horizontal text regions as input to the character recognition model improves the accuracy of identifying the production date inside the bottle cap at any angle, thereby increasing production efficiency and effectively helping enterprises improve product production quality and enhance their control over product quality. Attached Figure Description

[0070] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0071] Figure 1 This is a flowchart of a method for identifying the production date inside a bottle cap, provided by the present invention.

[0072] Figure 2 This is a schematic diagram of the adaptive feature fusion module;

[0073] Figure 3 This is a schematic diagram illustrating the classification and analysis of the text box direction in the angle adjuster; among which, Figure 3 (a) in the diagram represents a scenario where the text box is oriented to the upper left of the first case. Figure 3 (b) in the diagram represents a scenario where the text box is oriented to the lower left of the second case.

[0074] Figure 4 This is a diagram illustrating the detection of text boxes;

[0075] Figure 5 This is a schematic diagram of the text box segmentation results. Detailed Implementation

[0076] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0077] Example 1

[0078] Reference Figure 1 As shown, Figure 1 This is a flowchart of a method for identifying the production date inside a bottle cap, provided by the present invention; specifically including:

[0079] S1: Obtain the bottle cap image, preprocess the bottle cap image, and obtain the preprocessed bottle cap image;

[0080] S2: Using a text detection model, based on the preprocessed bottle cap image, obtain the coordinates of the four vertices of the first row of text boxes and the four vertices of the second row of text boxes in the bottle cap image, including:

[0081] S21: Input the preprocessed bottle cap image into the backbone network of the text detection model and output feature maps of the bottle cap image at various scales; wherein, the backbone network of the text detection model is a lightweight residual network ResNet18, which includes four residual blocks; the number of channels of the four residual blocks are 64, 128, 256 and 512 respectively;

[0082] S22: Input the feature maps of the bottle cap image at various scales into the neck network of the text detection module. Use the adaptive feature fusion module to fuse the features of two feature maps at adjacent scales respectively, and output the third fused feature map of the two feature maps at adjacent scales, such as... Figure 2 As shown, it includes:

[0083] The bottle cap image is input into the adjacent low-scale and high-scale feature maps, which are then fed into the initial feature fusion unit. The output is the first fused feature map of the two adjacent feature maps, and its expression is:

[0084] ;

[0085] in, This represents the first fused feature map of two feature maps at adjacent scales. This represents the low-scale feature map among two adjacent feature maps corresponding to the bottle cap image; This represents the high-scale feature map corresponding to two adjacent feature maps of the bottle cap image;

[0086] The first fused feature map is input into the channel attention unit, and the weights corresponding to the low-scale feature map and the weights corresponding to the high-scale feature map are output.

[0087] The low-scale feature map and its corresponding weights, along with the high-scale feature map and its corresponding weights, are input into the feature re-fusion unit. The output is the second fused feature map of the two feature maps at the current adjacent scales, and its expression is:

[0088] ;

[0089] in, This represents the second fusion feature map of the bottle cap image; This represents the low-scale feature map among two adjacent feature maps corresponding to the bottle cap image; This represents the high-scale feature map corresponding to two adjacent feature maps of the bottle cap image; This indicates the weights of the lower-scale feature maps in two adjacent feature maps at the current scale. This represents the weights of the higher-scale feature maps in two adjacent feature maps at the current scale.

[0090] The second fused feature map is input into the spatial attention unit, and the third fused feature map of the two feature maps at the current adjacent scale is output.

[0091] The third fused feature map of all two adjacent feature maps is input into the output layer of the neck network to output the final fused feature map of the bottle cap image.

[0092] S23: Input the final fused feature map of the bottle cap image into the head network of the text detection model, and output the predicted probability map corresponding to the bottle cap image; based on the predicted probability map corresponding to the bottle cap image, obtain the coordinates of the four vertices of the first row of text boxes and the four vertices of the second row of text boxes in the bottle cap image;

[0093] In one specific embodiment of the present invention, before performing operations S21-S23, the method further includes:

[0094] Acquire multiple bottle cap images and construct an image dataset;

[0095] Using an image annotation tool, the first and second lines of text in each bottle cap image in the image dataset are annotated to obtain the coordinates of the four vertices of the first and second line text boxes in each bottle cap image, and a coordinate information label set is constructed; based on the image dataset and its corresponding coordinate information label set, a text detection dataset is constructed.

[0096] The text detection dataset is preprocessed by adjusting all bottle cap images in the corresponding image dataset to 1280*720 pixels, and scaling all vertex coordinates in the coordinate information label set proportionally to obtain the preprocessed text detection dataset.

[0097] Based on the coordinate information labels of each bottle cap image in the preprocessed text detection dataset, for any pixel in each bottle cap image, if the current pixel is inside the text box, the probability value of the current pixel is 1; if the current pixel is outside the text box, the probability value of the current pixel is 0. This generates a true probability map for each bottle cap image. The probability value of each pixel is used as the true label of each pixel.

[0098] Scaling and dilution operations are performed on the true probability map of each bottle cap image to obtain the true threshold map of each bottle cap image, the expression of which is:

[0099] ;

[0100] in, Indicates the first True threshold map of bottle cap image; Indicates the first Area of ​​the true probability graph of the bottle cap image; Indicates the coefficient of shrinkage; Indicates the first The perimeter of the true probability map of the bottle cap image; the value of each pixel in each true threshold map is the threshold value for each pixel;

[0101] Using a differentiable binarization method, based on the true probability map and true threshold map of each bottle cap image, a true approximate binarized map of each bottle cap image is obtained, and its expression is:

[0102] ;

[0103] in, Indicates the first The true approximate binary image of the bottle cap image is in coordinates. Pixel value at; Indicates the first The true probability graph of the bottle cap image is shown on the coordinates. Pixel value at; Indicates the first The true threshold map of the bottle cap image is shown in the coordinates. Pixel value at; Represents a constant; if Then the true approximate binary map is in coordinates The pixel at this location belongs to the text box, and its coordinates are... The pixel at that location corresponds to the real label 1; if Then the true approximate binary map is in coordinates The pixel at this location is outside the text box; at this point, its coordinates are... The actual label corresponding to the pixel at that location is 0;

[0104] Based on the true probability map, true threshold map, and true approximate binary map of each bottle cap image, a set of true image labels is constructed.

[0105] Based on the preprocessed text detection dataset and the set of real image labels, a text detection data label set is constructed;

[0106] The text detection data label set is divided into a 70% text detection training set and a 30% text detection test set; before model training, each bottle cap image needs to be randomly rotated, randomly cropped and randomly scaled for data augmentation.

[0107] With the goal of minimizing the difference between the predicted probability and the true label for each pixel, a probabilistic graphical loss function is constructed, the expression of which is:

[0108] ;

[0109] in, This represents the set of all pixels in the probability graph; In the probability graph of prediction, the first... The probability value of a pixel being located in a text region; Represents the first in the true probability graph The true label of the nth pixel, if the nth pixel in the true probability map If a pixel is located within the text area, then ,otherwise ;

[0110] With the objective of minimizing the error between the predicted threshold map and the true threshold map, a threshold map loss function is constructed, the expression of which is:

[0111] ;

[0112] Specifically, a threshold map is obtained by dilating the probability map. This represents the set of all pixels in the threshold image; The first one represents the prediction threshold map. The threshold for each pixel; Represents the true threshold in the graph. The true threshold for each pixel;

[0113] With the objective of minimizing the difference between the predicted approximate binary map and the true approximate binary map, an approximate binary map loss function is constructed, the expression of which is:

[0114] ;

[0115] in, This represents the set of all pixels in an approximate binary image. Indicates the prediction of the first value in the approximate binary graph. The probability value of a pixel being located in a text region; Represents the first in a true approximation binary graph. The true label of the nth pixel, if the true label is approximated by the nth pixel in the binary image If a pixel is located within the text area, then ,otherwise ;

[0116] Based on the probabilistic graphical loss function, the threshold graphical loss function, and the approximate binary graphical loss function, a text detection loss function is constructed, the expression of which is:

[0117] ;

[0118] in, This represents the loss function of the text detection model; This represents the probabilistic graphical loss function; This represents the threshold plot loss function; This represents the loss function for an approximate binary graph. Indicates the threshold graph weights; Indicates the weights of the binary graph;

[0119] Using a set of text detection data labels, an initial text detection model is trained by minimizing the text detection loss function, resulting in a well-trained initial text detection model. The training process uses the Adam algorithm for gradient optimization and the F1 score to comprehensively evaluate the model performance. The trained initial text detection model is then used to perform text detection on a bottle cap image. The bottle cap image is input into the text detection model, which outputs the location information of all text regions in the bottle cap image.

[0120] In summary, the backbone of the text detection model uses a residual network RseNet18 as the feature extraction method to obtain features at different scales of the image. The residual blocks are progressively connected layer by layer, extracting feature maps at different scales. For example, using a 1280*720 pixel image from the training set as input, four residual blocks can extract feature maps with the following channel numbers and lengths and widths: (64, 320, 180), (128, 160, 90), (256, 80, 45), and (512, 40, 23), respectively. The neck section employs an FPN structure as the feature fusion method to enhance the multi-scale information of the extracted feature maps, effectively fusing features from different levels. Features are used, and the AF module is used instead of directly adding elements to enhance semantic information, resulting in fused features. The fused features contain high-level semantic features and low-level detail features, thereby improving detection accuracy. Before feature fusion, deep features need to be upsampled. For example, a feature map with dimensions (40, 23) needs to be upsampled to (80, 45) before being fed into the AF module. The head uses DBHead as the output module of the network model, outputting the position information of all text regions in the bottle cap image. When training the initial text detection model, both threshold maps and probability maps are used to improve detection accuracy, while only the probability map is used during model inference to improve inference speed.

[0121] S3: By performing a perspective transformation on the detected text box area, the detected text is kept horizontal; specifically, text boxes at any angle inside the bottle cap can be divided into two directions for discussion: the upper left direction in case one and the lower left direction in case two, such as... Figure 3 As shown in (a) and (b) above; specifically including:

[0122] Using an angle adjuster, in a clockwise direction from left to right, the coordinates of the four vertices of the first line of text boxes in the bottle cap image are sequentially labeled as the first coordinate point, the second coordinate point, the third coordinate point, and the fourth coordinate point. If the distance between the first and second coordinate points is less than or equal to the distance between the second and third coordinate points, then based on the vertex coordinates of the first line of text boxes in the bottle cap image, a perspective transformation algorithm is used, combined with a preset cutting size, to cut out the first line of text area image from the bottle cap image. For example... Figure 4 As shown; if the distance between the first coordinate point and the second coordinate point is greater than the distance between the second coordinate point and the third coordinate point, then the first coordinate point and the third coordinate point are interchanged. After obtaining the new coordinates of the four vertices of the first line of text box in the bottle cap image, perspective transformation is used, combined with the preset cutting size, to cut out the image of the first line of text area in the bottle cap image; wherein, the preset cutting size is 180*32.

[0123] Using the angle adjuster, the text area of ​​the second row in the bottle cap image is cut out based on the coordinates of the four vertices of the text box in the second row in the bottle cap image;

[0124] Based on S3, the cut text can always remain horizontal. That is, after the S3 operation, the text area image of each line in the cut bottle cap image is its corresponding corrected horizontal image, which can greatly improve the recognition accuracy of the character recognition model.

[0125] S4: Input the first and second line text regions from the bottle cap image into the character recognition model, and output the first and second line text sequences corresponding to the bottle cap image; wherein, the first line text sequence corresponding to the bottle cap image is the recognized production date inside the bottle cap;

[0126] The character recognition model adopts a CRNN architecture, including: convolutional layers, recurrent layers, and transcription layers, including:

[0127] The convolutional layer consists of six convolutional kernels of size 3 and one convolutional kernel of size 2 stacked together. It is used to extract the features corresponding to the first and second lines of text regions in the bottle cap image, respectively, to obtain the feature maps corresponding to the first and second lines of text regions in the bottle cap image. The convolutional layer uses multiple convolution and pooling operations to capture important image features and reduce the number of parameters, so that multiple convolutional layers can effectively extract important information features in the image. For example, for an input image of size 180*32 pixels, the stacked convolutional layers can extract a feature map with the number of channels and a length and width of (512, 1, 46).

[0128] The recurrent layer is a bidirectional long short-term memory network, used to perform sequence modeling on the feature maps corresponding to the first and second lines of text regions in the bottle cap image, respectively, to obtain the feature sequences corresponding to the first and second lines of text regions in the bottle cap image. The use of a bidirectional long short-term memory network to perform sequence modeling on the extracted features can capture the contextual relationships between characters, thereby establishing connections between contextual information. For example, based on a feature map with input (512, 1, 46), a feature map with dimensions (46, 14) can be output.

[0129] The transcription layer is used to map the feature sequences corresponding to the first and second lines of text regions in the bottle cap image to a preset character set, respectively, to generate the first and second line text sequences corresponding to the bottle cap image. The first line text sequence corresponds to the identified production date inside the bottle cap; the second line text sequence corresponds to the identified time information inside the bottle cap. The preset character set is limited to 14 character classes, including the character '-', i.e., '-0123456789:BC'. By restricting the encoding and decoding character set, unnecessary characters are discarded, thus avoiding unnecessary computational overhead and improving the model's inference speed. The transcription layer uses a fully connected network to map to the preset character set space and outputs the text sequences within the text regions.

[0130] Before performing the S3 operation, the following is also required:

[0131] Acquire multiple bottle cap images and construct an image dataset; cut the images according to a preset cutting size and the true coordinates of the four vertices of each row of text boxes in each bottle cap image to obtain the first row of text region images and the second row of text region images in each bottle cap image;

[0132] Using image annotation tools, the real text data and path source of the first and second lines of text information in each bottle cap image in the image dataset are annotated, and a real text label set is constructed; based on the text region images corresponding to each line of text in each bottle cap image in the image dataset and their corresponding real text label sets, a text recognition dataset is constructed.

[0133] The text recognition dataset is preprocessed by adjusting the text region images corresponding to each line of text in each bottle cap image to 180*32 pixels to obtain the preprocessed text recognition dataset.

[0134] The preprocessed text recognition dataset is divided into a 70% text recognition training set and a 30% text recognition test set. Before model training, the text region images corresponding to each row of bottle cap images need to be grayscaled to obtain grayscale images of the text region images corresponding to each row of bottle cap images.

[0135] Using the connection-based temporal classification loss function as the character recognition loss function, its expression is:

[0136] ;

[0137] in, An image representing the text region input to the character recognition model; This represents the actual character sequence corresponding to the text region image in the bottle cap image; This represents the text recognition training set; Represents probability; This represents the set of all possible paths to predict the true character sequence from the input text region image; This represents all possible paths to predict the true character sequence from the input text region image; the CTC loss, by introducing a "blank" mechanism, can effectively solve the problems of repeated characters and alignment in character recognition; after the initial character recognition model is trained, this model can accurately identify the text within the text box detected by the text detection model;

[0138] Using a character recognition training set, an initial character recognition model is trained by minimizing the character recognition loss function, resulting in a well-trained initial character recognition model. During training, the CTC loss function is used to train the initial character recognition model, and accuracy is used as a comprehensive performance metric to evaluate the character recognition model. Finally, the corrected horizontal image is used as input, and the character recognition model is used to accurately identify the production date inside the bottle cap, outputting the specific text content.

[0139] It should be noted that although the examples in this application are specifically for limited character sets and preset image sizes, these conditions should not be considered as limitations on the scope of application of this method. In fact, in the actual application of this method, if different character set limitations or other image size requirements are encountered, the technical solutions described in this application can still be adopted.

[0140] Example 2

[0141] This invention also provides a device for detecting and identifying the production date inside a bottle cap, which applies the aforementioned method for identifying the production date inside a bottle cap, including...

[0142] Image acquisition module: used to acquire bottle cap images and preprocess the bottle cap images to obtain preprocessed bottle cap images;

[0143] Text detection module: Used to obtain the coordinates of the four vertices of the first row of text boxes and the four vertices of the second row of text boxes in the preprocessed bottle cap image using a text detection model;

[0144] The rotation correction module uses an angle adjuster to sequentially assign the coordinates of the four vertices of the first line of text boxes in the bottle cap image to the first, second, third, and fourth coordinate points in a clockwise direction from left to right. If the distance between the first and second coordinate points is less than or equal to the distance between the second and third coordinate points, the first line of text area in the bottle cap image is cut out based on the vertex coordinates of the first line of text boxes using a perspective transformation algorithm and a preset cutting size. If the distance between the first and second coordinate points is greater than the distance between the second and third coordinate points, the first and third coordinate points are swapped to obtain new coordinates for the four vertices of the first line of text boxes in the bottle cap image. Then, perspective transformation is used, combined with a preset cutting size, to cut out the first line of text area in the bottle cap image. Finally, using the angle adjuster, the second line of text area in the bottle cap image is cut out based on the coordinates of the four vertices of the second line of text boxes.

[0145] Text recognition module: Used to obtain the first and second line text sequences corresponding to the bottle cap image based on the first and second line text regions in the bottle cap image using a text recognition model; wherein, the first line text sequence in the bottle cap image is the recognized production date inside the bottle cap;

[0146] The recording and alarm module is used to match the first line of text sequence in the bottle cap image with the date of production and the second line of text sequence with the time of production. If there is a mismatch in either line of text information, it will be recorded in time and an alarm will be sounded.

[0147] The device for detecting and recognizing the production date inside the bottle cap uses a cascaded approach to achieve end-to-end detection and recognition. This involves connecting the image acquisition module, text detection module, rotation correction module, text recognition module, and recording and alarm module in series. First, the text detection module acquires the coordinates of the four vertices of the first row of text boxes and the four vertices of the second row of text boxes in the bottle cap image. Then, based on these coordinates, a diagram is drawn as shown below. Figure 5The system first sets the border as shown. Then, it inputs the coordinates of the four vertices of the first and second line text boxes in the bottle cap image into the rotation correction module to cut out the text regions of each line in the bottle cap image, ensuring the text within the target text box is horizontal. Next, it inputs the text regions of each line in the bottle cap image into the character recognition model for recognition and outputs the corresponding text sequence. Finally, it compares the detected and recognized text sequence with the date and time of production. If it does not match the date and time, the system can record and issue a reminder for subsequent analysis and processing.

[0148] Example 3

[0149] The text detection model and character recognition model are converted to the ONNX general format to obtain general format text detection models and character recognition models. Model quantization is then performed on the general format text detection models and character recognition models to obtain compressed and optimized text detection models and character recognition models. The model quantization operation involves quantizing the 32-bit floating-point type (float32) parameters in the model to an 8-bit integer type (int8) to obtain a compressed and optimized model. The quantized model can significantly reduce computational complexity and file size, and is also easy to use across platforms.

[0150] In summary, this invention introduces an adaptive AF feature fusion module. To verify the effectiveness of the AF feature adaptive fusion module in text detection tasks, we conducted a comparative experiment on the bottle cap dataset, comparing the performance of the original model and the model after adding the AF module. The experimental results are shown in Table 1 below. The F1 score of the original model was 81.3%, while after introducing the AF feature adaptive fusion module, the F1 score of the model increased to 82.4%. This indicates that the AF module effectively improves the overall detection level of the model, and its overall performance is better than that of the original model.

[0151] Table 1 Experimental Results

[0152] method accuracy % Recall rate % F1 value % FPS DBNet 87.3 76.8 81.3 58 DBNet+AF 89.5 76.6 82.4 63

[0153] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for identifying the production date inside a bottle cap, characterized in that, include: Obtain the bottle cap image, preprocess the bottle cap image, and obtain the preprocessed bottle cap image; The preprocessed bottle cap image is input into the backbone network of the text detection model, and the output is the feature map of the bottle cap image at various scales. The feature maps at various scales of the bottle cap image are input into the neck network of the text detection module. An adaptive feature fusion module is then used to fuse the features of two feature maps at adjacent scales, outputting a third fused feature map of the two adjacent scales, including: The bottle cap image is fed into the adjacent low-scale and high-scale feature maps to the initial feature fusion unit, which outputs the first fused feature map of the two adjacent feature maps. The first fused feature map is then fed into the channel attention unit, which outputs the weights corresponding to the low-scale feature map and the high-scale feature map. The low-scale feature map and its corresponding weights, along with the high-scale feature map and its corresponding weights, are fed into the feature fusion unit again, which outputs the second fused feature map of the two adjacent feature maps. The second fused feature map is then fed into the spatial attention unit, which outputs the third fused feature map of the two adjacent feature maps. The third fused feature map of all two adjacent feature maps is input into the output layer of the neck network to output the final fused feature map of the bottle cap image. The final fused feature map of the bottle cap image is input into the head network of the text detection model, and the predicted probability map corresponding to the bottle cap image is output. Based on the predicted probability map corresponding to the bottle cap image, the coordinates of the four vertices of the first row of text boxes and the coordinates of the four vertices of the second row of text boxes in the bottle cap image are obtained. Using an angle adjuster, in a clockwise direction from left to right, the coordinates of the four vertices of the first line of text boxes in the bottle cap image are sequentially recorded as the first coordinate point, the second coordinate point, the third coordinate point, and the fourth coordinate point. If the distance between the first and second coordinate points is less than or equal to the distance between the second and third coordinate points, then based on the vertex coordinates of the first line of text boxes in the bottle cap image, a perspective transformation algorithm is used, combined with a preset cutting size, to cut out the first line of text area image in the bottle cap image. If the distance between the first and second coordinate points is greater than the distance between the second and third coordinate points, then the first and third coordinate points are swapped. After obtaining the new coordinates of the four vertices of the first line of text boxes in the bottle cap image, a perspective transformation is used, combined with a preset cutting size, to cut out the first line of text area image in the bottle cap image. Using the angle adjuster, the text area of ​​the second row in the bottle cap image is cut out based on the coordinates of the four vertices of the text box in the second row in the bottle cap image; Input the first and second line text regions from the bottle cap image into the character recognition model, and output the first and second line text sequences corresponding to the bottle cap image; wherein, the first line text sequence corresponding to the bottle cap image is the recognized production date inside the bottle cap.

2. The method for identifying the production date inside a bottle cap according to claim 1, characterized in that, The backbone network of the text detection model is a lightweight residual network ResNet18, which includes four residual blocks; the number of channels in the four residual blocks are 64, 128, 256, and 512, respectively.

3. The method for identifying the production date inside a bottle cap according to claim 1, characterized in that, The process involves inputting the adjacent low-scale and high-scale feature maps of the bottle cap image into the initial feature fusion unit, and outputting the first fused feature map of the two adjacent feature maps, the expression of which is: ； in, This represents the first fused feature map of two feature maps at adjacent scales. This represents the low-scale feature map among two adjacent feature maps corresponding to the bottle cap image; This represents the high-scale feature map corresponding to two adjacent feature maps of the bottle cap image.

4. The method for identifying the production date inside a bottle cap according to claim 1, characterized in that, The low-scale feature map and its corresponding weights, along with the high-scale feature map and its corresponding weights, are input into the feature re-fusion unit, which outputs a second fused feature map of the two feature maps at the current adjacent scales. Its expression is as follows: ； in, This represents the second fusion feature map of the bottle cap image; This represents the low-scale feature map among two adjacent feature maps corresponding to the bottle cap image; This represents the high-scale feature map corresponding to two adjacent feature maps of the bottle cap image; This indicates the weights of the lower-scale feature maps in two adjacent feature maps at the current scale. This represents the weights of the higher-scale feature maps in the two adjacent feature maps.

5. The method for identifying the production date inside a bottle cap according to claim 1, characterized in that, Also includes: Acquire multiple bottle cap images and construct an image dataset; Using an image annotation tool, the first and second lines of text in each bottle cap image in the image dataset are annotated to obtain the coordinates of the four vertices of the first and second line text boxes in each bottle cap image, and a coordinate information label set is constructed; based on the image dataset and its corresponding coordinate information label set, a text detection dataset is constructed. The text detection dataset is preprocessed by adjusting all bottle cap images in the corresponding image dataset to 1280*720 pixels, and scaling all vertex coordinates in the coordinate information label set proportionally to obtain the preprocessed text detection dataset. Based on the coordinate information labels of each bottle cap image in the preprocessed text detection dataset, for any pixel in each bottle cap image, if the current pixel is inside the text box, the probability value of the current pixel is 1; if the current pixel is outside the text box, the probability value of the current pixel is 0. This generates a true probability map for each bottle cap image. The probability value of each pixel is used as the true label of each pixel. Scaling and dilution operations are performed on the true probability map of each bottle cap image to obtain the true threshold map of each bottle cap image, the expression of which is: ； in, Indicates the first True threshold map of bottle cap image; Indicates the first Area of ​​the true probability graph of the bottle cap image; Indicates the coefficient of shrinkage; Indicates the first The perimeter of the true probability map of the bottle cap image; the value of each pixel in each true threshold map is the threshold value for each pixel; Using a differentiable binarization method, based on the true probability map and true threshold map of each bottle cap image, a true approximate binarized map of each bottle cap image is obtained, and its expression is: ； in, Indicates the first The true approximate binary image of the bottle cap image is in coordinates. Pixel value at; Indicates the first The true probability graph of the bottle cap image is shown on the coordinates. Pixel value at; Indicates the first The true threshold map of the bottle cap image is shown in the coordinates. Pixel value at; Represents a constant; if Then the true approximate binary map is in coordinates The pixel at this location belongs to the text box, and its coordinates are... The actual label corresponding to the pixel at that location is 1; if Then the true approximate binary map is in coordinates The pixel at this location is outside the text box; at this point, its coordinates are... The actual label corresponding to the pixel at that location is 0; Based on the true probability map, true threshold map, and true approximate binary map of each bottle cap image, a set of true image labels is constructed. Based on the preprocessed text detection dataset and the set of real image labels, a text detection data label set is constructed; The text detection data label set is divided into a 70% text detection training set and a 30% text detection test set; To minimize the difference between the predicted probability and the true label of each pixel, a probabilistic graphical loss function is constructed; to minimize the error between the predicted threshold map and the true threshold map, a threshold map loss function is constructed; to minimize the difference between the predicted approximate binary map and the true approximate binary map, an approximate binary map loss function is constructed; and based on the probabilistic graphical loss function, the threshold map loss function, and the approximate binary map loss function, a text detection loss function is constructed. Using the text detection data label set, the initial text detection model is trained by minimizing the text detection loss function, and the trained initial text detection model is used as the text detection model. Cut the image according to the preset cutting size and the actual coordinates of the four vertices of each line of text boxes in each bottle cap image to obtain the first line of text region image and the second line of text region image in each bottle cap image. Using image annotation tools, the real text data and path source of the first and second lines of text information in each bottle cap image in the image dataset are annotated, and a real text label set is constructed; based on the text region images corresponding to each line of text in each bottle cap image in the image dataset and their corresponding real text label sets, a text recognition dataset is constructed. The text recognition dataset is preprocessed by adjusting the text region images corresponding to each line of text in each bottle cap image to 180*32 pixels to obtain the preprocessed text recognition dataset. The preprocessed text recognition dataset was divided into a 70% text recognition training set and a 30% text recognition test set. The connection-time classification loss function is used as the character recognition loss function; using the character recognition training set, the initial character recognition model is trained by minimizing the character recognition loss function, and the trained initial character recognition model is used as the character recognition model.

6. The method for identifying the production date inside a bottle cap according to claim 5, characterized in that, The expression for the text detection loss function is: ； in, The loss function of the text detection model is used. This represents the probabilistic graphical loss function; This represents the threshold plot loss function; This represents the loss function for an approximate binary graph. Indicates the threshold graph weights; Indicates the weights of the binary graph; The expression for the probabilistic graphical loss function is: ； in, This represents the set of all pixels in the probability graph; In the probability graph of prediction, the first... The probability value of a pixel being located in a text region; In the true probability graph, the first... The true label of the nth pixel, if the nth pixel in the true probability map If a pixel is located in the text area, then ,otherwise ; The expression for the approximate binary graph loss function is: ； in, This represents the set of all pixels in an approximate binary image. Indicates the prediction of the first value in the approximate binary graph. The probability value of a pixel being located in a text region; Represents the first in a true approximation binary graph. The true label of the nth pixel, if the true label is approximated by the nth pixel in the binary image If a pixel is located in the text area, then ,otherwise ; The expression for the threshold map loss function is: ； Specifically, a threshold map is obtained by dilating the probability map. This represents the set of all pixels in the threshold image; The first one represents the prediction threshold map. The threshold for each pixel; Represents the true threshold in the graph. The true threshold for each pixel.

7. The method for identifying the production date inside a bottle cap according to claim 5, characterized in that, The expression for the character recognition loss function is: ； in, An image representing the text region input to the character recognition model; This represents the actual character sequence corresponding to the text region image in the bottle cap image; This represents the text recognition training set; Represents probability; This represents the set of all possible paths to predict the true character sequence from the input text region image; This represents all possible paths to predict the true character sequence from the input text region image.

8. The method for identifying the production date inside a bottle cap according to claim 1, characterized in that, The character recognition model adopts a CRNN architecture, including: convolutional layers, recurrent layers, and transcription layers; The convolutional layer is composed of six convolutional kernels of size 3 and one convolutional kernel of size 2 stacked together. It is used to extract the features corresponding to the first row of text regions and the second row of text regions in the bottle cap image, respectively, to obtain the feature map corresponding to the first row of text regions and the feature map corresponding to the second row of text regions in the bottle cap image. The recurrent layer is a bidirectional long short-term memory network, which is used to perform sequence modeling on the feature maps corresponding to the first row of text regions and the second row of text regions in the bottle cap image, respectively, to obtain the feature sequences corresponding to the first row of text regions and the second row of text regions in the bottle cap image. The transcription layer is used to map the feature sequences corresponding to the first and second lines of text regions in the bottle cap image to a preset character set, respectively, to generate the first and second line text sequences corresponding to the bottle cap image; wherein, the first line text sequence corresponding to the bottle cap image is the identified production date inside the bottle cap; and the second line text sequence corresponding to the bottle cap image is the identified time information inside the bottle cap.

9. A device for detecting and identifying the production date inside a bottle cap, characterized in that, The method for identifying the production date inside a bottle cap as described in any one of claims 1-8 includes: Image acquisition module: used to acquire bottle cap images and preprocess the bottle cap images to obtain preprocessed bottle cap images; Text detection module: Used to obtain the coordinates of the four vertices of the first row of text boxes and the four vertices of the second row of text boxes in the preprocessed bottle cap image using a text detection model; The rotation correction module uses an angle adjuster to sequentially assign the coordinates of the four vertices of the first line of text boxes in the bottle cap image to the first, second, third, and fourth coordinate points in a clockwise direction from left to right. If the distance between the first and second coordinate points is less than or equal to the distance between the second and third coordinate points, the first line of text area in the bottle cap image is cut out based on the vertex coordinates of the first line of text boxes using a perspective transformation algorithm and a preset cutting size. If the distance between the first and second coordinate points is greater than the distance between the second and third coordinate points, the first and third coordinate points are swapped to obtain new coordinates for the four vertices of the first line of text boxes in the bottle cap image. Then, perspective transformation is used, combined with a preset cutting size, to cut out the first line of text area in the bottle cap image. Finally, using the angle adjuster, the second line of text area in the bottle cap image is cut out based on the coordinates of the four vertices of the second line of text boxes. Text recognition module: Used to obtain the first and second line text sequences corresponding to the bottle cap image based on the first and second line text regions in the bottle cap image; wherein, the first line text sequence in the bottle cap image is the identified production date inside the bottle cap.

10. A device for detecting and identifying the production date inside a bottle cap according to claim 9, characterized in that, Also includes: The recording and alarm module is used to match the first line of text sequence in the bottle cap image with the date of production and the second line of text sequence with the time of production. If there is a mismatch in either line of text information, it will be recorded in time and an alarm will be sounded.

Images