A method and system for detecting and removing in-line three-phase codes

Abstract

A detection and rejection method and system of on-line three-stage codes, comprising the following steps: a camera acquires an image of a product, the processed image is input into an improved YOLO11-obb network model to obtain position information and a rotation angle of each character on the three-stage code; an Opencv module cuts and rotates the characters to a normal angle according to the position information and the rotation angle; after scaling the pictures of each character by using a padding method, the pictures are recombined into a new batch, which is input into an improved YOLO11-cls network model to output the classification index of each character; the position information of each character is sorted and compared with a standard text, and rejection information is sent to a rejection mechanism when an anomaly occurs; the rejection mechanism rejects unqualified products, and the system counts product detection data. The present application can effectively remove interference, can batch inference, detect faster, more stable, more accurate, fast deployment, high degree of automation, and can ensure stable rejection of unqualified products.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automation detection technology, and in particular to a detection and rejection method and system for three-phase codes on a production line. BACKGROUND

[0002] Three-phase code detection is an important link to ensure product traceability, compliance and quality in the food packaging and pharmaceutical industries. By automatically identifying production date, expiration date and batch number through OCR technology, key information can be monitored in real time to avoid misprinting, missing printing and blurring (unclear text, text breaks, text burrs) and other problems. At the same time, combined with bar code or two-dimensional code technology, three-phase code detection can also strengthen product anti-counterfeiting function, help batch management and quality traceability, and improve enterprise production efficiency and consumer trust.

[0003] The current three-phase code detection in the food packaging and pharmaceutical industries faces several technical difficulties: first, the production line usually needs to handle multiple types of products, and different materials and surface characteristics make it difficult to unify the lighting conditions, and traditional detection methods are insufficient in compatibility; second, the items may shake during transmission and coding, causing character distortion, including font stretching, distortion, blurring and other problems, increasing the difficulty of recognition. In addition, for rotating objects (such as coding on bottle caps), the fixed direction of characters further increases the complexity of detection, and special text direction detection algorithms must be introduced to solve these problems. In this context, deep learning technology can effectively adapt to diverse scenarios and has higher robustness to complex deformation and rotating characters, thereby significantly improving the accuracy and reliability of detection. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art, provide a detection and rejection method for three-phase codes on a production line with fast detection speed, high detection accuracy and strong compatibility, and a detection and rejection system implementing the method.

[0005] The present application is implemented by the following technical solutions:

[0006] A detection and rejection method for three-phase codes on a production line, comprising the following steps:

[0007] S1, when the product to be detected on the conveying line passes through the camera, triggering the camera to take a picture and obtaining the image of the three-phase code of the product, and processing the image;

[0008] S2, inputting the image obtained in step S1 into the improved YOLO11-obb network model trained, and identifying the position information and rotation angle of each character on the three-phase code through the improved YOLO11-obb network;

[0009] S3, the Opencv module cuts out each character from the image according to the position information and rotation angle of each character in step S2, rotates each character to the normal angle, and then scales the picture of each character to the specified size by padding;

[0010] S4, after the pictures of each character obtained in step S3 are combined into a new batch, the new batch is input into the improved YOLO11-cls network model which has been trained, and the classification index of each character is output;

[0011] S5, the classification index of each character is sorted according to the position information of each character in step S2, and the sorted text is compared with the standard text set in advance. If the two do not match, or there is a character with a recognition confidence lower than the set threshold, it is judged as unqualified three-phase code, and the unqualified information is sent to the rejection mechanism. If there is a character with a recognition confidence lower than the set threshold (such as 0.5) in a batch, the text is also judged as unqualified. The lower the confidence, the more blurred or deformed the character is, and the lower the confidence, the less output value.

[0012] S6, the rejection mechanism rejects the unqualified products according to the obtained rejection signal; the system counts the total number of products, the number of unqualified products, the rejection rate, the type of unqualified products and the number of each type.

[0013] Further, the training of the improved YOLO11-obb network and the improved YOLO11-cls network is automatic training, and the method is:

[0014] a. Obtain the three-phase code pictures of various products on the production line. The pictures should cover all possible products on the production line, including three-phase codes of different forms of the same product;

[0015] b. Use the SAM vision large model to extract the characters on the picture and record the position information of the characters to generate the position data set; then use the text classification model to classify and recognize the extracted characters and record the recognized characters to generate the character category data set, realizing automatic labeling and automatic generation of the data set; manual work only needs to check whether the automatically completed data set is correct, which greatly improves the debugging efficiency of OCR on the production line.

[0016] c. Train the improved YOLO11-obb network through the position data set, and train the improved YOLO11-cls network through the character category data set.

[0017] Further, the text classification model is a YOLO11X-CLS pre-training large model.

[0018] Further, the processing of the image in the S1 step includes normalization processing that compresses the image pixel range from 0-255 to 0-1.

[0019] Further, the improved YOLO11-obb network includes a backbone network part and a head network part.

[0020] The backbone network part includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module, and a SimAM attention mechanism module connected in sequence.

[0021] The head network part includes a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module, and an OBB module connected in sequence.

[0022] The implementation process of the backbone network part is as follows: after the input is extracted for preliminary features by the first Conv module [64, 3, 2], the feature enters the second Conv module [128, 3, 2] for down-sampling, and then continues to enter the first C3k2 module [256, False, 0.25] for further feature extraction; after the feature map is down-sampled by the third Conv module [256, 3, 2] at the P3 / 8 scale, the second C3k2 module [512, False, 0.25] is used to process the feature; after down-sampling to P4 / 16 by the fourth Conv module [512, 3, 2], the third C3k2 module [512, True] is used for deeper feature extraction; after down-sampling to P5 / 32 by the fifth Conv module [1024, 3, 2], the fourth C3k2 module [1024, True] is used for processing; the feature map is further processed by the SPPF module [1024, 5] to extract multi-scale features; after being processed by the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information.

[0023] The implementation process of the head network part is: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times (nn.Upsample) through the first upsampling module Upsample, and then the upsampled P5 is spliced with the P4 feature map through the first splicing module Concat; the spliced P4 feature map is processed by the fifth C3k2 module [512, False] and then output feature map is upsampled by 2 times (nn.Upsample) through the second upsampling module Upsample, to obtain the P4 upsampled feature map, and then the upsampled feature map is spliced with the P3 feature map through the second splicing module Concat; the spliced P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale, to obtain the downsampled P4 feature map; the downsampled P4 feature map is spliced with the P4 feature map processed by the fifth C3k2 module [512, False] through the third splicing module Concat, and then further extracted features through the seventh C3k2 module [512, False], and then downsampled by the seventh Conv module [512, 3, 2] to return to the P5 scale, to obtain the P5 downsampled P5 feature map; the downsampled P5 feature map is spliced with the P5 scale feature map of the backbone network part through the fourth splicing module Concat, and then processed by the eighth C3k2 module [1024, True] to output the final P5 scale feature; the P3, P4 and P5 scale feature maps are input into the OBB module, the OBB module completes the detection of the target with direction, and outputs the final detection result, including the position and rotation angle of each target.

[0024] Further, the improved YOLO11-cls network includes a backbone network part and a head network part.

[0025] The backbone network part includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module connected in sequence.

[0026] The head network part comprises, in sequence, a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and a Classify module.

[0027] The implementation process of the backbone network part is as follows: after the input is extracted with the first Conv module [64, 3, 2] to obtain preliminary features, the features enter the second Conv module [128, 3, 2] for down-sampling, and then continue to enter the first C3k2 module [256, False, 0.25] for further feature extraction; after the feature map is down-sampled by the third Conv module [256, 3, 2], at the P3 / 8 scale, the second C3k2 module [512, False, 0.25] is used to process the features; after down-sampling by the fourth Conv module [512, 3, 2] to P4 / 16, the third C3k2 module [512, True] is used for deeper feature extraction; after down-sampling by the fifth Conv module [1024, 3, 2] to P5 / 32, the fourth C3k2 module [1024, True] is used for processing; the feature map is further processed by the SPPF module [1024, 5] to extract multi-scale features; after being processed by the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information.

[0028] The implementation process of the head network part is: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times through the first upsampling module Upsample, and then the upsampled P5 is spliced with the P4 feature map through the first splicing module Concat; the spliced P4 feature map is processed by the fifth C3k2 module [512, False], and then the output feature map is upsampled by 2 times through the second upsampling module Upsample to obtain the P4 upsampled feature map; then the upsampled feature map is spliced with the P3 feature map through the second splicing module Concat; the spliced P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map; the downsampled P4 feature map is spliced with the P4 feature map processed by the fifth C3k2 module [512, False] through the third splicing module Concat, and then further extracted features through the seventh C3k2 module [512, False]; then downsampled by the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 downsampled P5 feature map; the downsampled P5 feature map is spliced with the P5 scale feature map of the backbone network part through the fourth splicing module Concat, and then processed by the eighth C3k2 module [1024, True] to output the final P5 scale feature; the feature maps of the P3, P4 and P5 scales are input into the Classify, and the Classify classifies the detection target to output the final detection result, including the categories corresponding to each target.

[0029] Further, the improved YOLO11-obb network and the improved YOLO11-cls network are both accelerated by tensorRT.

[0030] A detection and rejection system for three-stage codes on a production line is used to implement the detection and rejection method for three-stage codes on a production line. The system comprises a conveying line, a camera, a detection and analysis mechanism, and a rejection mechanism. The conveying line is used to convey products to be detected. The camera and the rejection mechanism are arranged on the side or above the conveying line, and the rejection mechanism is located downstream of the camera. The camera is used to detect and capture the products to be detected on the conveying line to obtain images with three-stage codes of the products, and transmit the images to the detection and analysis mechanism. The detection and analysis mechanism is connected with the camera and the rejection mechanism, respectively, and is used to receive the images uploaded by the camera, identify and judge the three-stage codes of the products on the images, send rejection information to the rejection mechanism, and statistically analyze the detection results. The rejection mechanism is used to reject the products corresponding to unqualified three-stage codes on the conveying line according to the received instructions.

[0031] The detection and analysis mechanism comprises a model automatic training module, an image receiving and processing module, an improved YOLO11-obb network module, an improved YOLO11-cls network module, an Opencv module and a data analysis and statistics module.

[0032] The model automatic training module is configured to automatically train the improved YOLO11-obb network and the improved YOLO11-cls network.

[0033] The image receiving and processing module is configured to receive images uploaded by a camera and carrying product three-phase codes, and to normalize, scale and splice the images.

[0034] The improved YOLO11-obb network module is configured to identify position information and rotation angles of characters in the images, and the SimAM attention mechanism is added to the improved YOLO11-obb network module.

[0035] The Opencv module is configured to crop the characters from the images according to the position information and rotation angles of the characters, and to rotate the characters to normal angles.

[0036] The improved YOLO11-cls network module is configured to batch classify and identify all characters in a batch, and the SimAM attention mechanism is added to the improved YOLO11-cls network module.

[0037] The data analysis and statistics module is configured to sort the identified characters according to the position information of the characters, compare the characters with a standard text, determine whether the characters are qualified or not, and count a total number of detected products, an unqualified number, a rejection rate, types of unqualified products and numbers of the types of unqualified products.

[0038] Further, a control display mechanism is further included, which is connected with the detection and analysis mechanism, configured to display operation interfaces of the model automatic training module and the data analysis and statistics module, so as to be operated by a user, and display results of automatic labeling and automatic statistics, so as to be viewed by the user.

[0039] Further, the model automatic training module comprises a picture acquisition unit, a position data set automatic generation unit, a character category data set automatic generation unit, an improved YOLO11-obb network training unit and an improved YOLO11-cls network training unit.

[0040] The picture acquisition unit is configured to acquire pictures to be trained by a camera.

[0041] The position data set automatic generation unit comprises a SAM visual large model, configured to automatically identify positions of characters in the pictures, automatically label and extract the characters, and automatically generate position data sets according to the positions of the characters.

[0042] The character category dataset automatic generation unit comprises a text classification model, which is used for classifying and identifying the extracted characters, and automatically generating a character category dataset according to the identification result.

[0043] The improved YOLO11-obb network training unit is used for inputting the data in the position dataset into the improved YOLO11-obb network for model training.

[0044] The improved YOLO11-cls network training unit is used for inputting the data in the character category dataset into the improved YOLO11-cls network for model training.

[0045] The three-stage code pictures on the production line are first detected for text position and rotation angle by the improved YOLO11-obb network, and each character is identified, and then all the characters are detected for classification and identification by the improved YOLO11-cls network through rotation correction and padding, so that the invalid area in the picture can be effectively removed, the interference factors can be eliminated, the batch inference mode is used, and compared with the mainstream rcnn single character detection mode, the method is more stable, and better recognition effect is obtained for the text with rotation; the SimAM attention mechanism module is added at the end of the backbone of the network YOLO11-obb and YOLO11-cls, so that the accuracy of the detection frame and the accuracy of the rotation angle after model inference can be effectively improved; batch inference, lightweight network and tensorRT model acceleration are used to further realize high-performance deployment, so that the inference speed is greatly improved; through the deployment of the automatic labeling and automatic dataset generation module, the automation of model training can be realized, the efficiency of model training is improved, and the compatibility of the model is improved; the model has strong universality, compared with an intelligent camera or a traditional algorithm, a set of model can be compatible with multiple products, and it is not necessary to debug each product again; even if the product is changed greatly and needs to be retrained, the training speed is greatly improved, and the deployment speed is fast; the complete detection and elimination process and structure can ensure stable elimination while ensuring high-precision detection, and there is no phenomenon of detection without elimination or elimination without detection. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The method flowchart of the embodiment of the application.

[0047] Figure 2 The example diagram of the position information identification of the character in the embodiment of the application.

[0048] Figure 3 The example diagram of the character after rotation correction in the embodiment of the application.

[0049] Figure 4An example diagram of a display interface controlled in a detection system in an embodiment of the present application.

[0050] Figure 5 A flowchart of a method for automatic training in an embodiment of the present application.

[0051] Figure 6 An example diagram of automatic labeling in an embodiment of the present application.

[0052] Figure 7 An architecture diagram of an improved lightweight network YOLO11-obb in an embodiment of the present application.

[0053] Figure 8 An architecture diagram of an improved lightweight network YOLO11-cls in an embodiment of the present application.

[0054] Figure 9 A framework diagram of a detection and rejection system in an embodiment of the present application.

[0055] Figure 10 A structural schematic diagram of a camera support in a detection and rejection system in an embodiment of the present application. DETAILED DESCRIPTION

[0056] A detection and rejection method for three-period codes on a production line, as shown in Figure 1 , includes the following steps:

[0057] S1, when the product to be detected on the conveying line passes through the camera, triggering the camera to take a picture, obtaining the image of the three-period code of the product, and processing the image. The test system is pre-set to detect the area, and the three-period code area of the product is set as the detection area. The camera is arranged beside or above the conveying line. The camera signal is obtained through the I / O board card, and when the camera signal is obtained, the image frame in the data stream is obtained. The image is processed, mainly normalized processing. The normalized processing compresses the image pixel range from 0-255 to 0-1.

[0058] S2, input the image obtained in S1 step into the improved lightweight network YOLO11-obb which has been trained, as shown in Figure 2 , the position information and the rotation angle of each character on the three-period code are recognized by the YOLO11-obb network.

[0059] S3, the Opencv module cuts out each character from the image according to the position information and the rotation angle of each character in S2 step, as shown in Figure 3 , and after rotating each character to a normal angle, the picture of each character is scaled to a specified size by padding method for use by the next network. After extracting each character, padding is used. This way can remove useless information in the image and put the text to be detected into the recognition network.

[0060] S4, after recombining the pictures of each character obtained in step S3 into a new batch, inputting the new batch into the improved lightweight network YOLO11-cls which has been trained, and outputting the classification indexes of each character.

[0061] Generally, there are multiple characters (including multiple lines of text) in an image. It is very wasteful to infer each character separately. In the batch inference mode, the pictures corresponding to each character are spliced into a batch to become a picture array, which is input into the network at one time. In this way, the network only needs to be identified once to obtain the recognition results of all text characters, including multiple lines of text, which improves the recognition speed and effect. Since each character has position information, there will be no confusion.

[0062] S5, sorting the classification indexes of each character according to the position information of each character in step S2, comparing the sorted text with the pre-set standard text, and if they do not match, such as missing printing, wrong printing, or there are characters with a classification recognition confidence lower than the set threshold, such as blur, font deformation, etc., it is judged as unqualified three-phase code, and the unqualified information is sent to the rejection mechanism, and the rejection signal is output through the I / O board card. If there is a character with a recognition confidence lower than the set threshold (such as 0.5) in a batch, the text is also determined as unqualified. A lower confidence indicates that the character is blurred or severely deformed, and a lower confidence means no output value.

[0063] In addition, the pre-set standard text can be automatically updated. For example, for the three-phase code of production date, market date, and shelf life, the system time can be called to update the pre-set text, such as setting the pre-set text as "market Q{today's time}", and the compared text is automatically generated as "market Q20241214".

[0064] S6, the rejection mechanism rejects the unqualified products according to the obtained rejection signal. Figure 4 The system can also monitor the total number of products detected, the number of unqualified products, the rejection rate, the yield, the type of unqualified products, and the number of each type of information in real time.

[0065] As shown in Figure 5 The training of the improved lightweight network YOLO11-obb and the improved lightweight network YOLO11-cls is automatic, and the method is as follows:

[0066] a. Obtain the three-phase code pictures of each product on the production line. The pictures should cover all possible products on the production line, including different three-phase code forms of the same product. The range covered by the picture set determines the scope of the final model. In order to adapt to as many products on the production line as possible without the need to retrain the model with the change of products, and in order to improve the accuracy of inference, the range of the picture set should be as wide as possible, and the number of pictures for each category should be as large as possible. The form of the three-phase code on different products will be different, such as some products only need to print the production date, some products need to print the production date and the on-sale date, some production dates need to be accurate to the specific time and minute, and some need to print the production batch number; for text, there are also font and size differences, and the range of text involves numbers, Chinese and English, symbols, etc.; for product packaging, such as glass bottles, plastic bottles, plastic bags, metal cans, etc., the differences in material and shape will also affect recognition. Therefore, as many pictures as possible should be covered during training.

[0067] b. Use the SAM visual large model to extract characters on the picture and record the position information of the characters to generate the position data set; then use the text classification model to classify and recognize the extracted characters and record the recognized characters to generate the character category data set, realizing automatic labeling and automatic generation of the data set. As shown in Figure 6 , the automatic labeling can be intuitively viewed. The SAM visual large model is an existing trained model, and running the model can give a text detection prompt and automatically frame the text.

[0068] For model training, data set labeling is a very time-consuming and labor-intensive work, and the quality of labeling also relates to the training effect of the model. The present scheme can identify text from pictures through a general existing model, and automatically classify the text with a large text classification model. In this embodiment, the text classification model uses YOLO11X-CLS pre-training large model. The YOLO11X-CLS pre-training large model needs to be trained in advance and can recognize most cases and quickly classify the data set. If you want to further reduce the false detection rate, you can add a model on the real production line for retraining. Training pictures on the real production line takes less time, but can improve the accuracy of detection.

[0069] The two large models are combined, when the model training is needed, a picture is input, only one button needs to be clicked, automatic labeling of the picture can be realized, all text positions are located, and the text in the picture is accurately recognized. A new sample is realized, and the data set is automatically prepared, and the artificial only needs to check whether the automatically prepared data set is correct, so that the debugging efficiency of the OCR in the production line is greatly improved. Artificial inspection can be carried out in the picture recognition process, or in the data set, each character in the data set corresponds to a folder, and whether other characters appear in the folder can be checked.

[0070] c. The improved lightweight network YOLO11-obb is trained through the position data set, and the improved lightweight network YOLO11-cls is trained through the character category data set. The two network models can be trained on the training platform.

[0071] The improved YOLO11-obb network and the improved YOLO11-cls network in the application mainly add S i m AM The attention mechanism module is used to strengthen the infusion of key information, and effectively improve the accuracy of the detection frame and the accuracy of the rotation angle after model reasoning.

[0072] Specifically, as Figure 7 The improved lightweight network YOLO11-obb includes a backbone network part backbone and a head network part head;

[0073] The backbone network part backbone includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, a SPPF module, a C2PSA module and a SimAM attention mechanism module connected in sequence;

[0074] The head network part head includes a first up-sampling module Upsample, a first splicing module Concat, a fifth C3k2 module, a second up-sampling module Upsample, a second splicing module Concat, a sixth C3k2 module, a sixth Conv module, a third splicing module Concat, a seventh C3k2 module, a seventh Conv module, a fourth splicing module Concat, an eighth C3k2 module and an OBB module connected in sequence.

[0075] The implementation process of the backbone network part is as follows: after the input is extracted by the first Conv module [64, 3, 2] to obtain preliminary features, the features enter the second Conv module [128, 3, 2] for down-sampling, and then enter the first C3k2 module [256, False, 0.25] for further feature extraction; after the feature map is down-sampled by the third Conv module [256, 3, 2], at the P3 / 8 scale, the second C3k2 module [512, False, 0.25] is used to process the features; after down-sampling by the fourth Conv module [512, 3, 2] to P4 / 16, the third C3k2 module [512, True] is used for deeper feature extraction; after down-sampling by the fifth Conv module [1024, 3, 2] to P5 / 32, the fourth C3k2 module [1024, True] is used for processing; the feature map is further processed by the SPPF module [1024, 5] to extract multi-scale features; after the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information. The Args parameter [64, 3, 2] corresponds to [channel, kernel, stride] respectively, 64 represents the output channel number of the layer, 3 represents the convolution kernel size k, and 2 represents the stride step length. The input channel number of each layer is the output channel number of the previous layer by default.

[0076] The implementation process of the head network part head is: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times (nn.Upsample) through the first upsampling module Upsample, and then the upsampled P5 is spliced with the P4 feature map through the first splicing module Concat; the spliced P4 feature map is processed by the fifth C3k2 module [512, False], and then the output feature map is upsampled by 2 times (nn.Upsample) through the second upsampling module Upsample to obtain the P4 upsampled feature map, and then the upsampled feature map is spliced with the P3 feature map through the second splicing module Concat; the spliced P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map; the downsampled P4 feature map is spliced with the P4 feature map processed by the fifth C3k2 module [512, False] through the third splicing module Concat, and then further extracted features through the seventh C3k2 module [512, False], and then downsampled by the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 downsampled P5 feature map; the downsampled P5 feature map is spliced with the P5 scale feature map of the backbone network part through the fourth splicing module Concat, and then processed by the eighth C3k2 module [1024, True] to output the final P5 scale feature; the feature maps of the P3, P4 and P5 scales are input into the OBB module, the OBB module completes the detection of the target with direction, and outputs the final detection result, including the position and rotation angle of each target.

[0077] As Figure 8 , the improved lightweight network YOLO11-cls includes a backbone network part backbone and a head network part head;

[0078] The backbone network part backbone includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module connected in sequence.

[0079] The head network part head includes a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and a Classify module connected in sequence.

[0080] The implementation process of the backbone network part backbone is as follows: after the input is extracted by the first Conv module [64, 3, 2] to obtain preliminary features, the features enter the second Conv module [128, 3, 2] for down-sampling, and then continue to enter the first C3k2 module [256, False, 0.25] for further feature extraction; after the feature map is down-sampled by the third Conv module [256, 3, 2], at the P3 / 8 scale, the second C3k2 module [512, False, 0.25] is used to process the features; after down-sampling by the fourth Conv module [512, 3, 2] to P4 / 16, the third C3k2 module [512, True] is used for deeper feature extraction; after down-sampling by the fifth Conv module [1024, 3, 2] to P5 / 32, the fourth C3k2 module [1024, True] is used for processing; the feature map is further processed by the SPPF module [1024, 5] to extract multi-scale features; after being processed by the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information.

[0081] The implementation process of the head network part head is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times through the first upsampling module Upsample, and then the upsampled P5 is spliced with the P4 feature map through the first splicing module Concat; the spliced P4 feature map is processed through the fifth C3k2 module [512, False], and then the output feature map is upsampled by 2 times through the second upsampling module Upsample to obtain the P4 upsampled feature map, and then the upsampled feature map is spliced with the P3 feature map through the second splicing module Concat; the spliced P3 feature map is processed through the sixth C3k2 module [256, False], and then downsampled through the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map; the downsampled P4 feature map is spliced with the P4 feature map processed through the fifth C3k2 module [512, False] through the third splicing module Concat, and then further extracted features through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 downsampled P5 feature map; the downsampled P5 feature map is spliced with the P5 scale feature map of the backbone network part through the fourth splicing module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final P5 scale feature; the P3, P4 and P5 scale feature maps are input into the Classify, and the Classify classifies the detection target to output the final detection result, including the categories corresponding to each target.

[0082] The improved lightweight network YOLO11-obb and the improved lightweight network YOLO11-cls are both accelerated through tensorRT.

[0083] A detection and rejection system for three-stage codes on a production line is used to implement the detection and rejection method for three-stage codes on a production line as described above. Figure 9, including a conveying line for conveying products to be detected, a camera and a rejection mechanism arranged on the side or above the conveying line respectively, and the rejection mechanism is located downstream of the camera, the camera is used for detecting and shooting the products to be detected on the conveying line to obtain an image with a three-phase code of the product, and the image is transmitted to the detection analysis mechanism; the detection analysis mechanism is connected with the camera and the rejection mechanism respectively, used for receiving the image uploaded by the camera, identifying and judging the three-phase code of the product on the image, sending rejection information to the rejection mechanism, and statistically analyzing the detection results; the rejection mechanism is used for rejecting the product corresponding to the unqualified three-phase code on the conveying line according to the received instructions. The camera is installed on a camera support 1, and the camera shoots the side or top surface of the product. As shown in Figure 10 , a light source 2 and other facilities need to be configured on the support.

[0084] The rejection mechanism is used for rejecting the product on the conveying line, which can be selected and designed according to the specific structure and layout of the conveying line, and the size and weight of the product, such as a cylinder rejection structure, including a rejection cylinder, a rotating shaft and a rejection plate, the rejection plate is arranged on the side of the conveying line, the rejection plate is connected to the rotating shaft, the rejection cylinder is connected to the rotating shaft through a connecting rod structure, the extension and retraction of the rejection cylinder drives the rotating shaft to rotate, and drives the rejection plate to rotate, pushing the product on the conveying line away from the conveying line, the rejection cylinder can also be replaced by a motor, and the rejection plate is driven to rotate by the rotation of the motor to reject the product; it can also be a gas blowing rejection mode, which blows the product away from the conveying line.

[0085] The detection analysis mechanism includes a model automatic training module, an image receiving and processing module, an improved YOLO11-obb network module, an improved YOLO11-cls network module, an Opencv module and a data analysis and statistics module.

[0086] The model automatic training module is used for automatically training the improved YOLO11-obb network and the improved YOLO11-cls network.

[0087] The image receiving and processing module is used for receiving the image with the three-phase code of the product uploaded by the camera, and pre-normalizing the image in the input network model, scaling each character image extracted to a specified size to adapt to the next model, and splicing each character image into a batch for batch inference.

[0088] The improved YOLO11-obb network module is used for identifying the position information and rotation angle of each character in the image, and the SimAM attention mechanism is added.

[0089] The Opencv module is configured to crop each character from the image according to the position information and rotation angle of each character, and rotate each character to a normal angle.

[0090] The improved YOLO11-cls network module is configured to batch classify and identify all characters in a batch, and the SimAM attention mechanism is added.

[0091] The data analysis and statistics module is configured to sort the identified characters according to their position information, compare them with the standard text, determine whether they are qualified or not, and count the total number of detected products, the number of unqualified products, the rejection rate, the types of unqualified products, and the number of each type of information.

[0092] As Figure 4 , the control display mechanism is connected to the detection and analysis mechanism, and is configured to display the operation interface of the model automatic training module and the data analysis and statistics module, so that the user can operate and display the automatic labeling and automatic statistics results. The user can intuitively view the detection results and statistical data, such as the number of NG products, the yield, etc. The control display mechanism can also be connected to the camera to display the pictures transmitted by the camera.

[0093] The model automatic training module includes a picture acquisition unit, a position data set automatic generation unit, a character category data set automatic generation unit, an improved YOLO11-obb network training unit, and an improved YOLO11-cls network training unit.

[0094] The picture acquisition unit is configured to acquire the pictures to be trained through the camera; the position data set automatic generation unit includes a SAM visual large model, which is configured to automatically identify the character positions on the pictures, automatically label and extract the characters, and automatically generate the position data set according to the positions of the characters.

[0095] The character category data set automatic generation unit includes a text classification model, which is configured to classify and identify the extracted characters, and automatically generate the character category data set according to the identification results. The text classification model of the embodiment is a YOLO11X-CLS pre-training large model.

[0096] The improved YOLO11-obb network training unit is configured to input the data in the position data set into the improved YOLO11-obb network for model training.

[0097] The improved YOLO11-cls network training unit is configured to input the data in the character category data set into the improved YOLO11-cls network for model training.

[0098] The detection speed is fast, the detection precision is high, and the reliability is good. It has been proved that after the detection system of the application is configured with an RTX3060 graphics card, it only needs 35-40 ms to detect a picture, the FPS can reach 25-28, and the number of products that can be stably detected per minute is 1000+(the index of the industry, the number of detection per minute); the actual production detection rate, the leakage rate of unqualified three-phase codes due to missing printing and wrong printing is 0, and the unqualified three-phase codes due to font blur and font deformation only have a small amount of leakage, less than one ten-thousandth, and the false detection rate is less than one thousandth.

[0099] The above detailed description is specific to the feasible embodiments of the application, which is not used to limit the patent scope of the application, and any equivalent implementation or change without departing from the application shall be included in the patent scope of the application.

Claims

1. A method for detecting and rejecting in-line third stage codes, characterized by, The method comprises the following steps: S1, when the product to be detected on the conveying line passes through the camera, triggering the camera to take a picture, obtaining an image of the three-phase code of the product, and processing the image; S2, inputting the image obtained in step S1 into the improved YOLO11-obb network model which has been trained, and obtaining the position information and rotation angle of each character on the three-phase code through the improved YOLO11-obb network; S3, the Opencv module cuts out each character from the image according to the position information and rotation angle of each character in step S2, and rotates each character to a normal angle, and then scales the picture of each character to a specified size by padding; S4, after the pictures of each character obtained in step S3 are combined into a new batch, the new batch is input into the improved YOLO11-cls network model which has been trained, and the classification index of each character is output; S5, sorting the classification index of each character according to the position information of each character in step S2, comparing the sorted text with the standard text set in advance, and if the two do not match or there is a character with a classification recognition confidence lower than a set threshold, it is judged as an unqualified three-phase code, and the unqualified information is sent to the rejection mechanism; S6, the rejection mechanism rejects the unqualified product according to the obtained rejection signal; the system counts the total number of detected products, the number of unqualified products, the rejection rate, the type of unqualified products and the number of each type.

2. The method of claim 1, wherein the method is characterized by: The training of the improved YOLO11-obb network and the improved YOLO11-cls network is automatic, and the method is: a. obtaining three-phase code pictures of various products on the production line, including three-phase codes of different forms of the same product; b. using the SAM visual large model to extract characters on the picture, recording the position information of the characters, generating a position data set; then using a text classification model to classify and identify the extracted characters, recording the identified characters, generating a character category data set, realizing automatic labeling and automatic generation of the data set; c. training the improved YOLO11-obb network through the position data set, and training the improved YOLO11-cls network through the character category data set.

3. The method for detecting and removing three-phase codes on a production line according to claim 2, characterized in that, The text classification model is a YOLO11X-CLS pre-trained large model.

4. The method of claim 1, wherein the method is characterized by: The image processing in step S1 includes normalization processing, which compresses the image pixel range from 0-255 to 0-1.

5. The method of claim 1, wherein the method is characterized by: The improved YOLO11-obb network comprises a backbone network part and a head network part; The backbone network part comprises a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module connected in sequence; The head network part comprises, in sequence, a first upsample module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsample module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and an OBB module; The implementation process of the backbone network part is as follows: after the input is extracted with the first Conv module [64, 3, 2] to obtain preliminary features, the features enter the second Conv module [128, 3, 2] for down-sampling, and then continue to enter the first C3k2 module [256, False, 0.25] for further feature extraction; after the feature map is down-sampled by the third Conv module [256, 3, 2], at the P3 / 8 scale, the second C3k2 module [512, False, 0.25] is used to process the features; after down-sampling by the fourth Conv module [512, 3, 2] to P4 / 16, the third C3k2 module [512, True] is used for deeper feature extraction; after down-sampling by the fifth Conv module [1024, 3, 2] to P5 / 32, the fourth C3k2 module [1024, True] is used for processing; the feature map is further processed by the SPPF module [1024, 5] to extract multi-scale features; after being processed by the C2PSA module, the SimAM attention mechanism module [1024] is used to focus on key information; The implementation process of the head network part is that: the P5 feature map output from the SimAM attention mechanism module is up-sampled by 2 times through the first up-sampling module Upsample, and then the up-sampled P5 feature map is spliced with the P4 feature map through the first splicing module Concat; the spliced feature map is processed through the fifth C3k2 module [512, False], and then the output feature map is up-sampled by 2 times through the second up-sampling module Upsample to obtain the P4 up-sampled feature map; then the up-sampled feature map is spliced with the P3 feature map through the second splicing module Concat; the spliced P3 feature map is processed through the sixth C3k2 module [256, False], and then down-sampled through the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the down-sampled P4 feature map; the down-sampled P4 feature map is spliced with the P4 feature map processed through the fifth C3k2 module [512, False] through the third splicing module Concat, and then further extracted features through the seventh C3k2 module [512, False], and then down-sampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 down-sampled P5 feature map; the down-sampled P5 feature map is spliced with the P5 scale feature map of the backbone network part through the fourth splicing module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final P5 scale feature; the feature maps of the P3, P4 and P5 scales are input into the OBB module, the OBB module completes the detection of the target with direction, and outputs the final detection result, including the position and rotation angle of each target.

6. The method of claim 1, wherein the method is characterized by: The improved YOLO11-cls network comprises a backbone network part and a head network part; The backbone network part comprises a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module connected in sequence; The head network part comprises a first up-sampling module Upsample, a first splicing module Concat, a fifth C3k2 module, a second up-sampling module Upsample, a second splicing module Concat, a sixth C3k2 module, a sixth Conv module, a third splicing module Concat, a seventh C3k2 module, a seventh Conv module, a fourth splicing module Concat, an eighth C3k2 module and a Classify module connected in sequence; The head network part comprises a first up-sampling module Upsample, a first splicing module Concat, a fifth C3k2 module, a second up-sampling module Upsample, a second splicing module Concat, a sixth C3k2 module, a sixth Conv module, a third splicing module Concat, a seventh C3k2 module, a seventh Conv module, a fourth splicing module Concat, an eighth C3k2 module and a Classify module connected in sequence; The implementation process of the backbone network part is: after the input is extracted by the first Conv module [64, 3, 2] to obtain preliminary features, the features enter the second Conv module [128, 3, 2] for down sampling, and then continue to enter the first C3k2 module [256, False, 0.25] for further feature extraction; after the feature map is down sampled by the third Conv module [256, 3, 2], at the P3 / 8 scale, the second C3k2 module [512, False, 0.25] is used to process the features; after being down sampled to P4 / 16 by the fourth Conv module [512, 3, 2], the third C3k2 module [512, True] is used for deeper feature extraction; after being down sampled to P5 / 32 by the fifth Conv module [1024, 3, 2], the fourth C3k2 module [1024, True] is used for processing; the feature map further passes through the SPPF module [1024, 5] to extract multi-scale features; after passing through the C2PSA module, the SimAM attention mechanism module [1024] is used to focus on key information; The implementation process of the head network part is: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times through the first upsampling module Upsample, and then the upsampled P5 feature map is spliced with the P4 feature map through the first splicing module Concat; the spliced P4 feature map is processed through the fifth C3k2 module [512, False], and then the output feature map is upsampled by 2 times through the second upsampling module Upsample to obtain the P4 upsampled feature map; then the upsampled feature map is spliced with the P3 feature map through the second splicing module Concat; the spliced P3 feature map is processed through the sixth C3k2 module [256, False], and then downsampled through the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map; the downsampled P4 feature map is spliced with the P4 feature map processed through the fifth C3k2 module [512, False] through the third splicing module Concat, and then further extracted features through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 downsampled P5 feature map; the downsampled P5 feature map is spliced with the P5 scale feature map of the backbone network part through the fourth splicing module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final P5 scale feature; the feature maps of the P3, P4 and P5 scales are input into the Classify, and the Classify classifies the detection target to output the final detection result, including the categories corresponding to each target.

7. The method of claim 1, wherein the method is characterized by: The improved YOLO11-obb network and the improved YOLO11-cls network are both model accelerated through tensorRT.

8. A system for detecting and rejecting a line-produced tertiary code, which is used to implement the method for detecting and rejecting a line-produced tertiary code according to any one of claims 1 to 7, characterized in that, The detection analysis mechanism is connected with the camera and the rejection mechanism, used for receiving the image uploaded by the camera, identifying and judging the product three-phase code on the image, sending rejection information to the rejection mechanism, and statistically analyzing the detection result; the rejection mechanism is used for rejecting the product corresponding to the unqualified three-phase code on the conveying line according to the received instruction; The detection analysis mechanism includes a model automatic training module, an image receiving and processing module, an improved YOLO11-obb network module, an improved YOLO11-cls network module, an Opencv module, and a data analysis and statistics module. The model automatic training module is used for automatically training the improved YOLO11-obb network and the improved YOLO11-cls network; The image receiving and processing module is used for receiving images uploaded by a camera and carrying product three-phase codes, and performing normalization processing, scaling and splicing on the images; The improved YOLO11-obb network module is used for identifying position information and a rotation angle of each character on the images, and the SimAM attention mechanism is added to the improved YOLO11-obb network module; The Opencv module is used for cutting each character from the images according to the position information and the rotation angle of each character, and rotating each character to a normal angle; The improved YOLO11-cls network module is used for batch classification and identification of all characters in a batch, and the SimAM attention mechanism is added to the improved YOLO11-cls network module; The data analysis and statistics module is used for sorting the identified characters according to the position information, comparing the characters with a standard text, judging whether the characters are qualified or not, and counting a total number of detected products, an unqualified number, a rejection rate, types of unqualified products and quantities of the types.

9. The system for detecting and rejecting a line of three phase codes of claim 8, wherein, The control display mechanism is further included, which is connected with the detection and analysis mechanism, and is used for displaying operation interfaces of the model automatic training module and the data analysis and statistics module, so as to be operated by a user, and displaying automatic labeling and automatic statistical results, so as to be viewed by the user.

10. The system for detecting and rejecting a line of three phase codes of claim 8, wherein, The model automatic training module includes a picture acquisition unit, a position data set automatic generation unit, a character category data set automatic generation unit, an improved YOLO11-obb network training unit and an improved YOLO11-cls network training unit; The picture acquisition unit is used for acquiring pictures to be trained through a camera; The position data set automatic generation unit includes a SAM visual large model, which is used for automatically identifying character positions on the pictures, automatically labeling and extracting the characters, and automatically generating a position data set according to the positions of the characters; The character category data set automatic generation unit includes a text classification model, which is used for classifying and identifying the extracted characters, and automatically generating a character category data set according to a result of the identification; The improved YOLO11-obb network training unit is used for inputting data in the position data set into the improved YOLO11-obb network to perform model training; The improved YOLO11-cls network training unit is used for inputting data in the character category data set into the improved YOLO11-cls network to perform model training.

Images