Method and system for detecting and removing three-phase codes on production line

Abstract

A method and system for detecting and removing three-phase codes on a production line, and the method comprises the following steps: obtaining an image of a product through a camera, inputting the processed image into an improved YOLO11-obb network model, and obtaining the position information and rotation angle of each character on the three-phase codes; the Opencv module cuts the character according to the position information and the rotation angle, and rotates the character to a normal angle; after the pictures of all the characters are zoomed in a padding mode, all the pictures are recombined into a new batch, the new batch is input into the improved YOLO11-cls network model, and a classification index of all the characters is output; sorting is carried out according to the position information of the characters, comparison with a standard text is carried out, and rejection information is sent to the rejection mechanism when abnormity occurs; and the rejecting mechanism rejects unqualified products, and the system counts product detection data. According to the method, interference can be effectively removed, batch reasoning can be achieved, detection is faster, more stable and more accurate, the deployment speed is high, the automation degree is high, and it can be guaranteed that unqualified products are stably removed.

Description

Technical Field

[0001] The present invention relates to the field of automated detection technology, and in particular to a method and system for detecting and rejecting three-phase codes on a production line. Background Art

[0002] The three-phase code detection is an important part of ensuring product traceability, compliance and quality in the food packaging and pharmaceutical industries. Through the automatic identification of key information such as production date, expiration date and batch number through OCR technology, the production line can be monitored in real time to avoid problems such as misprinting, missing printing and blurring (unclear text, text breakpoints, text rough edges). At the same time, combined with barcode or QR code technology, the three-phase code detection can also enhance product anti-counterfeiting functions, assist batch management and quality traceability, and improve corporate production efficiency and consumer trust.

[0003] Currently, the detection of phase III codes in the food packaging and pharmaceutical industries faces multiple technical difficulties: First, the production line usually needs to process multiple types of products, and different materials and surface characteristics make it difficult to unify lighting conditions, and traditional detection methods are insufficient in compatibility; second, the items may shake during the transmission and coding process, resulting in character deformation, including font stretching, distortion, blurring and other problems, which increases the difficulty of recognition. In addition, for rotating objects (such as codes on bottle caps), the non-fixed direction of the characters further increases the complexity of detection, and a special text direction detection algorithm must be introduced to solve these problems. In this context, deep learning technology can effectively adapt to diverse scenarios through special training, and has higher robustness to complex deformations and rotated characters, thereby significantly improving the accuracy and reliability of detection. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for detecting and rejecting three-phase codes on a production line with fast detection speed, high detection accuracy and strong compatibility, as well as a detection and rejection system for implementing the method.

[0005] The present invention is achieved through the following technical solutions:

[0006] A method for detecting and removing three-phase codes on a production line comprises the following steps:

[0007] S1. When the product to be inspected on the conveyor line passes the camera, the camera is triggered to take a photo, an image of the product's three-phase code is obtained, and the image is processed;

[0008] S2, input the image obtained in step S1 into the trained improved YOLO11-obb network model, and obtain the position information and rotation angle of each character on the three-phase code through the improved YOLO11-obb network recognition;

[0009] S3, the Opencv module cuts out each character from the image according to the position information and rotation angle of each character in step S2, rotates each character to a normal angle, and then scales the image of each character to a specified size using padding;

[0010] S4, after recombining the images of each character obtained in step S3 into a new batch, input it into the trained improved YOLO11-cls network model, and output the classification index of each character;

[0011] S5. Sort the classification index of each character according to the position information of each character in step S2, and compare the sorted text with the pre-set standard text. If the two do not match, or there are characters whose classification recognition confidence is lower than the set threshold, it is judged as an unqualified three-phase code, and the unqualified information is sent to the rejection mechanism; if there are characters in a batch whose recognition confidence is lower than the set threshold (such as 0.5), the text is also judged as unqualified. Low confidence means that the characters are blurred or severely deformed, and lower confidence means no output value.

[0012] S6. The rejection mechanism rejects unqualified products based on the rejection signals obtained; the system counts the total number of products tested, the number of unqualified products, the rejection rate, the types of unqualified products and the number of each type.

[0013] Furthermore, the training of the improved YOLO11-obb network and the improved YOLO11-cls network are both automated training, and the method is as follows:

[0014] a. Obtain the three-phase code pictures of various products on the production line. The pictures should cover all possible products on the production line, including different forms of three-phase codes for similar products;

[0015] b. Use the SAM visual model to extract characters from the image, record the position information of the characters, and generate a position data set. Then use the text classification model to classify and recognize the extracted characters, record the recognized characters, and generate a character category data set to achieve automatic labeling and automatic generation of the data set. Manual work only needs to check whether the automatically generated data set is correct, which greatly improves the debugging efficiency of OCR on the production line.

[0016] c. The improved YOLO11-obb network is trained using the position dataset, and the improved YOLO11-cls network is trained using the character category dataset.

[0017] Furthermore, the text classification model is a YOLO11X-CLS pre-trained large model.

[0018] Furthermore, the processing of the image in step S1 includes normalization processing, and the normalization processing compresses the image pixel range from 0-255 to 0-1.

[0019] Furthermore, the improved YOLO11-obb network includes a backbone network part and a head network part;

[0020] The backbone network part includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module which are connected in sequence;

[0021] The head network part includes a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and an OBB module connected in sequence;

[0022] The implementation process of the backbone network part is as follows: after the input passes through the first Conv module [64, 3, 2] to extract preliminary features, the features enter the second Conv module [128, 3, 2] for downsampling, and then continue to enter the first C3k2 module [256, False, 0.25] to further extract features; after the feature map passes through the third Conv module [256, 3, 2] for downsampling, the second C3k2 module [512, False, 0.25] is used to process the features on the P3 / 8 scale; and then passes through the fourth After the Conv module [512,3,2] is downsampled to P4 / 16, the third C3k2 module [512,True] is used to perform deeper feature extraction; then it is downsampled to P5 / 32 by the fifth Conv module [1024,3,2] and processed by the fourth C3k2 module [1024,True]; the feature map is then passed through the SPPF module [1024,5] to extract multi-scale features; after passing through the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information;

[0023] The implementation process of the head network part is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times (nn.Upsample) by the first upsampling module Upsample, and then the upsampled P5 and P4 feature maps are spliced ​​by the first concatenation module Concat; the spliced ​​P4 feature map is processed by the fifth C3k2 module [512, False], and the output feature map is then upsampled by 2 times (nn.Upsample) by the second upsampling module Upsample to obtain the upsampled feature map of P4, and then the upsampled feature map is concatenated with the feature map of P3 by the second concatenation module Concat; the spliced ​​P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale , obtain the downsampled P4 feature map; concatenate the downsampled P4 feature map with the P4 feature map processed by the fifth C3k2 module [512, False] through the third concatenation module Concat, further extract features through the seventh C3k2 module [512, False], and then downsample through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 downsampled P5 feature map; concatenate the downsampled P5 feature map with the feature map of the P5 scale of the backbone network through the fourth concatenation module Concat, and then process through the eighth C3k2 module [1024, True] to output the final features of the P5 scale; input the feature maps of the three scales of P3, P4, and P5 into the OBB module, the OBB module completes the detection of targets with directions, and outputs the final detection results, including the position and rotation angle of each target.

[0024] Furthermore, the improved YOLO11-cls network includes a backbone network part and a head network part;

[0025] The backbone network part includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module which are connected in sequence;

[0026] The head network part includes a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and a Classify module connected in sequence;

[0027] The implementation process of the backbone network part is as follows: after the input passes through the first Conv module [64, 3, 2] to extract preliminary features, the features enter the second Conv module [128, 3, 2] for downsampling, and then continue to enter the first C3k2 module [256, False, 0.25] to further extract features; after the feature map passes through the third Conv module [256, 3, 2] for downsampling, the second C3k2 module [512, False, 0.25] is used to process the features on the P3 / 8 scale; and then passes through the fourth After the Conv module [512,3,2] is downsampled to P4 / 16, the third C3k2 module [512,True] is used to perform deeper feature extraction; then it is downsampled to P5 / 32 by the fifth Conv module [1024,3,2] and processed by the fourth C3k2 module [1024,True]; the feature map is then passed through the SPPF module [1024,5] to extract multi-scale features; after passing through the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information;

[0028] The implementation process of the head network part is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times by the first upsampling module Upsample, and then the upsampled P5 and P4 feature maps are spliced ​​by the first concatenation module Concat; the spliced ​​P4 feature map is processed by the fifth C3k2 module [512, False], and the output feature map is then upsampled by 2 times by the second upsampling module Upsample to obtain the upsampled feature map of P4, and then the upsampled feature map is spliced ​​with the feature map of P3 by the second concatenation module Concat; the spliced ​​P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map ; The downsampled P4 feature map is concatenated with the P4 feature map processed by the fifth C3k2 module [512, False] through the third concatenation module Concat, and then further features are extracted through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 feature map after P5 downsampling; The downsampled P5 feature map is concatenated with the feature map of the P5 scale of the backbone network through the fourth concatenation module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final features of the P5 scale; The feature maps of the three scales of P3, P4 and P5 are input into the Classify, Classify classifies the detection targets, and outputs the final detection results, including the categories corresponding to each target.

[0029] Furthermore, the improved YOLO11-obb network and the improved YOLO11-cls network are both accelerated by tensorRT.

[0030] A detection and rejection system for three-phase codes on a production line is used to implement the detection and rejection method for the three-phase codes on the above-mentioned production line, comprising a conveyor line, a camera, a detection and analysis mechanism and a rejection mechanism, wherein the conveyor line is used to convey products to be detected, the camera and the rejection mechanism are respectively arranged on the side or above the conveyor line, and the rejection mechanism is located downstream of the camera, the camera is used to detect and photograph the products to be detected on the conveyor line to obtain images with the product three-phase codes, and transmit the images to the detection and analysis mechanism; the detection and analysis mechanism is respectively connected to the camera and the rejection mechanism, and is used to receive the images uploaded by the camera, identify and judge the product three-phase codes on the images, and send rejection information to the rejection mechanism, as well as perform statistical analysis on the detection results; the rejection mechanism is used to reject products corresponding to unqualified three-phase codes on the conveyor line according to the received instructions.

[0031] The detection and analysis mechanism includes a model automatic training module, an image receiving and processing module, an improved YOLO11-obb network module, an improved YOLO11-cls network module, an Opencv module, and a data analysis and statistics module.

[0032] The model automatic training module is used to automatically train the improved YOLO11-obb network and the improved YOLO11-cls network.

[0033] The image receiving and processing module is used to receive images with product phase III codes uploaded by the camera, and to perform normalization, scaling and splicing on the images.

[0034] The improved YOLO11-obb network module is used to identify the position information and rotation angle of each character on the image, and a SimAM attention mechanism is added to it.

[0035] The Opencv module is used to cut out each character from the image according to the position information and rotation angle of each character, and rotate each character to a normal angle.

[0036] The improved YOLO11-cls network module is used for batch classification and recognition of all characters in a batch, to which the SimAM attention mechanism is added.

[0037] The data analysis and statistics module is used to sort the recognized characters according to their position information, and compare them with standard texts to determine whether they are qualified or not, and to count the total number of tested products, the number of unqualified products, the rejection rate, the types of unqualified products and the number of each type.

[0038] Furthermore, it also includes a control display mechanism, which is connected to the detection and analysis mechanism and is used to display the operating interface of the model automatic training module and the data analysis and statistics module for users to operate, and display the results of automatic labeling and automatic statistics for users to view.

[0039] Furthermore, the model automatic training module includes a picture acquisition unit, a position data set automatic generation unit, a character category data set automatic generation unit, an improved YOLO11-obb network training unit, and an improved YOLO11-cls network training unit.

[0040] The picture acquisition unit is used to acquire the picture to be trained through a camera.

[0041] The position data set automatic generation unit includes a SAM visual large model, which is used to automatically identify the character positions on the image, automatically mark and extract the characters, and automatically generate a position data set according to the character positions.

[0042] The character category data set automatic generation unit includes a text classification model, which is used to classify and identify the extracted characters and automatically generate a character category data set according to the identification results.

[0043] The improved YOLO11-obb network training unit is used to input the data in the position data set into the improved YOLO11-obb network for model training.

[0044] The improved YOLO11-cls network training unit is used to input data in the character category data set into the improved YOLO11-cls network for model training.

[0045] The present invention first detects the text position and rotation angle of the three-phase code image on the production line through the improved yolo11-obb network, so that each character can be identified separately, and then all the characters are classified and identified at one time through the improved yolo11-cls network through rotation correction and padding. It can effectively remove invalid areas in the image and eliminate interference factors. The batch reasoning method is more stable than the mainstream rcnn single character detection method, and has better recognition effect for rotated text; adding the SimAM attention mechanism module at the end of the backbone of the network yolo11-obb and yolo11-cls can effectively improve the model reasoning post-detection The accuracy of the frame measurement and the accuracy of the rotation angle; batch reasoning, lightweight networks, and tensorRT model acceleration are used to further achieve high-performance deployment, which greatly improves the reasoning speed; by deploying automatic labeling and automatic data set generation modules, the automation of model training can be achieved, the efficiency of model training and the compatibility of the model can be improved, and the model is highly versatile. Compared with smart cameras or traditional algorithms, a set of models can be compatible with multiple products, and each product does not need to be re-debugged. Even if retraining is required due to major changes in new products, the training speed is greatly improved and the deployment speed is fast; the complete detection and rejection process and structure can ensure stable rejection while ensuring high-precision detection, and there is no phenomenon of detection without rejection or non-detection rejection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0047] Figure 2 This is an example diagram of character position information recognition in an embodiment of the present invention.

[0048] Figure 3 This is an example diagram of a character after normal rotation in an embodiment of the present invention.

[0049] Figure 4This is an example diagram of a control display interface in a detection system in an embodiment of the present invention.

[0050] Figure 5 Flow chart of the method for automated training in an embodiment of the present invention.

[0051] Figure 6 This is an example diagram of automatic labeling in an embodiment of the present invention.

[0052] Figure 7 The diagram is an architecture diagram of the improved lightweight network YOLO11-obb in an embodiment of the present invention.

[0053] Figure 8 The diagram is an architecture diagram of the improved lightweight network YOLO11-cls in an embodiment of the present invention.

[0054] Fig. 9 It is a framework diagram of a detection and rejection system according to an embodiment of the present invention.

[0055] Fig.10 It is a schematic diagram of the structure of a camera bracket in a detection and rejection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] A method for detecting and eliminating three-phase codes on a production line, such as Figure 1 As shown, the following steps are included:

[0057] S1. When the product to be inspected on the conveyor line passes the camera, the camera is triggered to take a photo, obtain the image of the product's three-phase code, and process the image. The test system pre-sets the inspection area, sets the product's three-phase code area as the inspection area, sets the camera to the side or above the conveyor line, and obtains the camera signal through the I / O board. When the camera signal is obtained, the image frame in the data stream is obtained, and the image is processed, mainly normalizing the image. Normalization compresses the image pixel range from 0-255 to 0-1.

[0058] S2. Input the image obtained in step S1 into the trained improved lightweight network YOLO11-obb, such as Figure 2 As shown in the figure, the position information and rotation angle of each character on the three-phase code are obtained through YOLO11-obb network recognition.

[0059] S3, Opencv module according to the position information and rotation angle of each character in step S2, such as Figure 3 , crop each character from the image, rotate each character to a normal angle, and then use padding to scale the image of each character to a specified size for use by the next network. After each character is extracted and then padded, this method can remove useless information from the image and put the text to be detected into the recognition network.

[0060] S4. After recombining the images of each character obtained in step S3 into a new batch, input them into the trained improved lightweight network YOLO11-cls, and output the classification index of each character.

[0061] Generally, there will be multiple characters (including multiple lines of text) in an image. It is a waste of computing resources to infer each character separately. Batch inference is adopted to piece together the pictures corresponding to each character into a batch and turn it into a picture array. The picture array enters and exits the network at one time. In this way, the network only needs to recognize once to obtain the recognition results of all text characters, including multiple lines of text, which improves the recognition speed and effect. Since each character has corresponding position information, there will be no confusion.

[0062] S5. Sort the classification index of each character according to the position information of each character in step S2, and compare the sorted text with the pre-set standard text. If the two do not match, such as missing or wrong printing, or there are characters whose confidence in classification recognition is lower than the set threshold, such as blur, font deformation, etc., it is judged as an unqualified three-phase code, and the unqualified information is sent to the rejection mechanism, and the rejection signal is output through the I\O board. If there are characters with recognition confidence lower than the set threshold (such as 0.5) in a batch, the text is also judged as unqualified. Low confidence means that the characters are blurry or severely deformed, and lower confidence means no output value.

[0063] In addition, the preset standard text can be automatically updated. For example, for three-phase codes such as production date, launch date, and shelf life, the system time can be called to update. For example, if the preset text is set to "launch Q{today's time}", the compared text will automatically generate the time of the day "launch Q20241214".

[0064] S6, the rejection mechanism rejects unqualified products according to the rejection signal obtained; Figure 4 The system counts the total number of products tested, the number of unqualified products, the rejection rate, the yield rate, the type of unqualified products and the number of each type, and can perform real-time monitoring.

[0065] like Figure 5 As shown, the training of the improved lightweight network YOLO11-obb and the improved lightweight network YOLO11-cls are both automated training, and the method is:

[0066] a. Obtain the three-phase code images of each product on the production line. The images should cover all possible products on the production line, including different three-phase code forms of similar products. The scope of the picture set determines the scope of application of the final model. In order to adapt to the various products on the production line as much as possible without retraining the model as the product is replaced, and in order to improve the accuracy of reasoning, the scope of the picture set should be as wide as possible, and each category of pictures should be as many as possible. The form of the three-phase code on different products will be different. For example, some products only need to print the production date, some products need to print the listing date in addition to the production date, some production dates need to be accurate to the specific time and minute, and some need to print the production batch number; for text, there are also differences in fonts and sizes, and the range of text involves numbers, Chinese and English, symbols, etc.; for product packaging, such as glass bottles, plastic bottles, plastic bags, metal cans and other types, differences in materials and shapes will also affect recognition. Therefore, these pictures should be covered as much as possible during training.

[0067] b. Use the SAM visual model to extract characters from the image, record the position information of the characters, and generate a position data set; then use the text classification model to classify and recognize the extracted characters, record the recognized characters, and generate a character category data set to achieve automatic annotation and automatic generation of data sets. Figure 6 As shown in the figure, you can intuitively view the automatic annotation situation. The SAM visual model is an existing trained model. Running this model can give detection text prompts and automatically select text.

[0068] For model training, data set annotation is a very time-consuming and labor-intensive task, and the quality of annotation is also related to the training effect of the model. This solution can identify text from pictures through an existing general model, and automatically classify text with a large text classification model. In this embodiment, this text classification model uses the YOLO11X-CLS pre-trained, large model. The YOLO11X-CLS pre-trained large model needs to be trained in advance, can recognize most situations, and can quickly classify data sets. If you want to further reduce the false detection rate, you can add the model on the real production line for retraining. Training pictures on the real production line takes less time, but it can improve the accuracy of detection.

[0069] Combining these two large models, when model training is needed, input a picture and click a button to automatically annotate the picture, locate all the text positions, and accurately identify what the text is. When a new sample comes, the data set is automatically prepared, and the manual inspection only needs to check whether the automatically prepared data set is correct, which greatly improves the debugging efficiency of OCR on the production line. Manual inspection can be performed during the image recognition process or in the data set. Each character in the data set corresponds to a folder, and it can be checked in the folder to see if other text appears.

[0070] c. Train the improved lightweight network YOLO11-obb with the position dataset, and train the improved lightweight network YOLO11-cls with the character category dataset. You can train the two network models with one click on the training platform.

[0071] The improved YOLO11-obb network and the improved YOLO11-cls network in the present invention mainly add S at the end of the backbone network part of the original YOLO11-obb network and the YOLO11-cls network. i m AM The attention mechanism module is used to strengthen the infusion of key information and effectively improve the accuracy of the detection box and the rotation angle after model reasoning.

[0072] Specifically, Figure 7 , the improved lightweight network YOLO11-obb includes a backbone network part backbone and a head network part head;

[0073] The backbone network includes the first Conv module, the second Conv module, the first C3k2 module, the third Conv module, the second C3k2 module, the fourth Conv module, the third C3k2 module, the fifth Conv module, the fourth C3k2 module, the SPPF module, the C2PSA module and the SimAM attention mechanism module, which are connected in sequence;

[0074] The head network part head includes the first upsampling module Upsample, the first concatenation module Concat, the fifth C3k2 module, the second upsampling module Upsample, the second concatenation module Concat, the sixth C3k2 module, the sixth Conv module, the third concatenation module Concat, the seventh C3k2 module, the seventh Conv module, the fourth concatenation module Concat, the eighth C3k2 module and the OBB module, which are connected in sequence.

[0075] The implementation process of the backbone network is as follows: after the input passes through the first Conv module [64, 3, 2] to extract preliminary features, the features enter the second Conv module [128, 3, 2] for downsampling, and then continue to enter the first C3k2 module [256, False, 0.25] to further extract features; the feature map passes through the third Conv module [256, 3, 2] for downsampling, and then uses the second C3k2 module [512, False, 0.25] to process features on the P3 / 8 scale; and then passes through the After the fourth Conv module [512,3,2] is downsampled to P4 / 16, the third C3k2 module [512,True] is used for deeper feature extraction; then the fifth Conv module [1024,3,2] is downsampled to P5 / 32, and the fourth C3k2 module [1024,True] is used for processing; the feature map passes through the SPPF module [1024,5] to extract multi-scale features; after passing through the C2PSA module, the SimAM attention mechanism module

[1024] is used to focus on key information. Args parameters [64,3,2] correspond to [channel, kernel, stride], 64 represents the number of output channels of this layer, 3 represents the convolution kernel size k, 2 represents the stride step size, and the number of input channels per layer is the output channel number of the previous layer by default.

[0076] The implementation process of the head network part head is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times (nn.Upsample) by the first upsampling module Upsample, and then the upsampled P5 and P4 feature maps are spliced ​​by the first concatenation module Concat; the spliced ​​P4 feature map is processed by the fifth C3k2 module [512, False], and the output feature map is then upsampled by 2 times (nn.Upsample) by the second upsampling module Upsample to obtain the upsampled feature map of P4, and then the upsampled feature map is spliced ​​with the feature map of P3 by the second concatenation module Concat; the spliced ​​P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale. degrees, and obtain the downsampled P4 feature map; the downsampled P4 feature map is concatenated with the P4 feature map processed by the fifth C3k2 module [512, False] through the third concatenation module Concat, and then further features are extracted through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2], back to the P5 scale, to obtain the P5 downsampled P5 feature map; the downsampled P5 feature map is concatenated with the feature map of the P5 scale of the backbone network through the fourth concatenation module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final feature of the P5 scale; the feature maps of the three scales of P3, P4 and P5 are input into the OBB module, the OBB module completes the detection of targets with directions, and outputs the final detection results, including the position and rotation angle of each target.

[0077] like Figure 8 , the improved lightweight network YOLO11-cls includes a backbone network part backbone and a head network part head;

[0078] The backbone network includes the first Conv module, the second Conv module, the first C3k2 module, the third Conv module, the second C3k2 module, the fourth Conv module, the third C3k2 module, the fifth Conv module, the fourth C3k2 module, the SPPF module, the C2PSA module and the SimAM attention mechanism module, which are connected in sequence;

[0079] The head network part head includes the first upsampling module Upsample, the first concatenation module Concat, the fifth C3k2 module, the second upsampling module Upsample, the second concatenation module Concat, the sixth C3k2 module, the sixth Conv module, the third concatenation module Concat, the seventh C3k2 module, the seventh Conv module, the fourth concatenation module Concat, the eighth C3k2 module and the Classify module, which are connected in sequence.

[0080] The implementation process of the backbone network is as follows: after the input passes through the first Conv module [64, 3, 2] to extract preliminary features, the features enter the second Conv module [128, 3, 2] for downsampling, and then continue to enter the first C3k2 module [256, False, 0.25] to further extract features; the feature map passes through the third Conv module [256, 3, 2] for downsampling, and then uses the second C3k2 module [512, False, 0.25] to process features on the P3 / 8 scale; and then passes through the After the fourth Conv module [512,3,2] is downsampled to P4 / 16, the third C3k2 module [512,True] is used to perform deeper feature extraction; then it is downsampled to P5 / 32 through the fifth Conv module [1024,3,2] and processed by the fourth C3k2 module [1024,True]; the feature map then passes through the SPPF module [1024,5] to extract multi-scale features; and after passing through the C2PSA module, the key information is focused on through the SimAM attention mechanism module

[1024] .

[0081] The implementation process of the head network part head is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times by the first upsampling module Upsample, and then the upsampled P5 and P4 feature maps are spliced ​​by the first concatenation module Concat; the spliced ​​P4 feature map is processed by the fifth C3k2 module [512, False], and the output feature map is upsampled by 2 times by the second upsampling module Upsample to obtain the upsampled feature map of P4, and then the upsampled feature map is spliced ​​with the feature map of P3 by the second concatenation module Concat; the spliced ​​P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map. Figure; the downsampled P4 feature map is concatenated with the P4 feature map processed by the fifth C3k2 module [512, False] through the third concatenation module Concat, and then the features are further extracted through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 feature map after P5 downsampling; the downsampled P5 feature map is concatenated with the feature map of the P5 scale of the backbone network through the fourth concatenation module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final features of the P5 scale; the feature maps of the three scales of P3, P4 and P5 are input into the Classify, Classify classifies the detection targets, and outputs the final detection results, including the categories corresponding to each target.

[0082] The above-mentioned improved lightweight network YOLO11-obb and improved lightweight network YOLO11-cls are both accelerated by tensorRT.

[0083] A detection and rejection system for three-phase codes on a production line is used to implement the detection and rejection method for three-phase codes on the production line, such as Fig. 9, including a conveyor line, a camera, a detection and analysis mechanism, a rejection mechanism and a control and display mechanism. The conveyor line is used to convey products to be detected. The camera and the rejection mechanism are respectively arranged beside or above the conveyor line, and the rejection mechanism is located downstream of the camera. The camera is used to detect and photograph the products to be detected on the conveyor line to obtain an image with a three-phase code of the product, and transmit the image to the detection and analysis mechanism; the detection and analysis mechanism is respectively connected to the camera and the rejection mechanism, and is used to receive the image uploaded by the camera, identify and judge the three-phase code of the product on the image, and send the rejection information to the rejection mechanism, as well as perform statistical analysis on the detection results; the rejection mechanism is used to reject the products corresponding to the unqualified three-phase code on the conveyor line according to the received instructions. The camera is installed on the camera bracket 1, and the camera takes pictures of the side or top surface of the product, which can be as follows Fig.10 As shown, the bracket also needs to be equipped with facilities such as light source 2.

[0084] The function of the rejection mechanism is to reject the products on the conveyor line. It can be selected and designed according to the specific structure and layout of the conveyor line, as well as the size and weight of the product. For example, it can be a cylinder rejection structure, including a rejection cylinder, a rotating shaft and a rejection plate. The rejection plate is arranged on the side of the conveyor line, and the rejection plate is connected to the rotating shaft. The rejection cylinder is connected to the rotating shaft through a connecting rod structure. The extension and contraction of the rejection cylinder drives the rotating shaft to rotate, which drives the rejection plate to rotate and pushes the products on the conveyor line away from the conveyor line. The rejection cylinder can also be replaced by a motor, which drives the rejection plate to rotate by the rotation of the motor to reject the product. It can also be an air-blowing rejection method, in which the product is blown away from the conveyor line by air.

[0085] The detection and analysis mechanism includes a model automatic training module, an image receiving and processing module, an improved YOLO11-obb network module, an improved YOLO11-cls network module, an Opencv module, and a data analysis and statistics module.

[0086] The model automatic training module is used to automatically train the improved YOLO11-obb network and the improved YOLO11-cls network.

[0087] The image receiving and processing module is used to receive images with product phase III codes uploaded by the camera, and to pre-normalize the images input into the network model, scale the extracted character images to a specified size to facilitate the next model, and splice the character images into a batch for batch reasoning.

[0088] The improved YOLO11-obb network module is used to identify the position information and rotation angle of each character on the image, and a SimAM attention mechanism is added to it.

[0089] The Opencv module is used to cut out each character from the image according to the position information and rotation angle of each character, and rotate each character to a normal angle.

[0090] The improved YOLO11-cls network module is used for batch classification and recognition of all characters in a batch, to which the SimAM attention mechanism is added.

[0091] The data analysis and statistics module is used to sort the recognized characters according to their position information, and compare them with the standard text to determine whether they are qualified or not, and to count the total number of tested products, the number of unqualified products, the rejection rate, the types of unqualified products and the number of each type.

[0092] like Figure 4 The control and display mechanism is connected to the detection and analysis mechanism, and is used to display the operation interface of the model automatic training module and the data analysis and statistics module for users to operate, and display the results of automatic labeling and automatic statistics. Users can intuitively view the detection results and statistical data, such as the number of NGs and yield of products. The control and display mechanism can also be connected to a camera to display pictures transmitted by the camera.

[0093] The model automatic training module includes a picture acquisition unit, a position data set automatic generation unit, a character category data set automatic generation unit, an improved YOLO11-obb network training unit, and an improved YOLO11-cls network training unit.

[0094] The image acquisition unit is used to acquire the image to be trained through a camera; the position data set automatic generation unit includes a SAM visual large model, which is used to automatically identify the character position on the image, automatically mark and extract the characters, and automatically generate a position data set according to the character position.

[0095] The character category data set automatic generation unit includes a text classification model, which is used to classify and recognize the extracted characters, and automatically generate a character category data set according to the recognition result. The text classification model of this embodiment is a YOLO11X-CLS pre-trained large model.

[0096] The improved YOLO11-obb network training unit is used to input the data in the position data set into the improved YOLO11-obb network for model training.

[0097] The improved YOLO11-cls network training unit is used to input data in the character category data set into the improved YOLO11-cls network for model training.

[0098] The detection speed of the present invention is fast, the detection accuracy is high, and the reliability is good. Practice has proved that after the detection system of the present invention is configured with an RTX3060 graphics card, it only takes 35-40ms to detect and process a picture, the FPS can reach 25-28, and 1000+ products can be stably detected per minute (industrial indicator, the number of detections per minute); the actual production detection rate is 0 for unqualified three-phase codes caused by missed printing or wrong printing, and there is only a small amount of missed detection for unqualified three-phase codes caused by blurred fonts and deformed fonts, less than one in ten thousand, and the false detection rate is less than one in a thousand.

[0099] The above detailed description is a specific description of a feasible embodiment of the present invention. The embodiment is not intended to limit the patent scope of the present invention. Any equivalent implementation or modification that does not deviate from the present invention should be included in the patent scope of this case.

Claims

1. A method for detecting and removing three-phase codes on a production line, characterized in that: The steps include: S1. When the product to be inspected on the conveyor line passes the camera, the camera is triggered to take a photo, an image of the product's three-phase code is obtained, and the image is processed; S2, input the image obtained in step S1 into the trained improved YOLO11-obb network model, and obtain the position information and rotation angle of each character on the three-phase code through the improved YOLO11-obb network recognition; S3, the Opencv module cuts out each character from the image according to the position information and rotation angle of each character in step S2, rotates each character to a normal angle, and then scales the image of each character to a specified size using padding; S4, after recombining the images of each character obtained in step S3 into a new batch, input it into the trained improved YOLO11-cls network model, and output the classification index of each character; S5, sorting the classification index of each character according to the position information of each character in step S2, and comparing the sorted text with the preset standard text. If the two do not match, or there is a character whose confidence of classification recognition is lower than the set threshold, it is judged as an unqualified three-phase code, and the unqualified information is sent to the rejection mechanism; S6. The rejection mechanism rejects unqualified products based on the rejection signals obtained; the system counts the total number of products tested, the number of unqualified products, the rejection rate, the types of unqualified products and the number of each type.

2. A method for detecting and removing three-phase codes on a production line according to claim 1, characterized in that: The training of the improved YOLO11-obb network and the improved YOLO11-cls network are both automatic training, and the method is as follows: a. Obtain the three-phase code images of various products on the production line, including different forms of three-phase codes for similar products; b. Use the SAM visual model to extract characters from the image, record the position information of the characters, and generate a position data set; then use the text classification model to classify and recognize the extracted characters, record the recognized characters, generate a character category data set, and realize automatic annotation and automatic generation of the data set; c. The improved YOLO11-obb network is trained using the position dataset, and the improved YOLO11-cls network is trained using the character category dataset.

3. A method for detecting and removing three-phase codes on a production line according to claim 2, characterized in that: The text classification model is a YOLO11X-CLS pre-trained large model.

4. A method for detecting and removing three-phase codes on a production line according to claim 1, characterized in that: The image processing in step S1 includes a normalization process, wherein the normalization process compresses the image pixel range from 0-255 to 0-1.

5. The method for detecting and removing three-phase codes on a production line according to claim 1 is characterized in that: The improved YOLO11-obb network includes a backbone network part and a head network part; The backbone network part includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module which are connected in sequence; The head network part includes a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and an OBB module connected in sequence; The implementation process of the backbone network part is as follows: after the input passes through the first Conv module [64, 3, 2] to extract preliminary features, the features enter the second Conv module [128, 3, 2] for downsampling, and then continue to enter the first C3k2 module [256, False, 0.25] to further extract features; after the feature map passes through the third Conv module [256, 3, 2] for downsampling, the second C3k2 module [512, False, 0.25] is used to process the features on the P3 / 8 scale; and then passes through the fourth After the Conv module [512,3,2] is downsampled to P4 / 16, the third C3k2 module [512,True] is used to perform deeper feature extraction; then it is downsampled to P5 / 32 by the fifth Conv module [1024,3,2] and processed by the fourth C3k2 module [1024,True]; the feature map is then passed through the SPPF module [1024,5] to extract multi-scale features; after passing through the C2PSA module, the SimAM attention mechanism module [1024] is used to focus on key information; The implementation process of the head network part is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times by the first upsampling module Upsample, and then the upsampled P5 feature map is spliced ​​with the P4 feature map by the first concatenation module Concat; the spliced ​​feature map is processed by the fifth C3k2 module [512, False], and the output feature map is upsampled by 2 times by the second upsampling module Upsample to obtain the P4 upsampled feature map, and then the upsampled feature map is spliced ​​with the P3 feature map by the second concatenation module Concat; the spliced ​​P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map ; The downsampled P4 feature map is concatenated with the P4 feature map processed by the fifth C3k2 module [512, False] through the third concatenation module Concat, and then further features are extracted through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 feature map after P5 downsampling; The downsampled P5 feature map is concatenated with the feature map of the P5 scale of the backbone network through the fourth concatenation module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final feature of the P5 scale; The feature maps of the three scales of P3, P4 and P5 are input into the OBB module, the OBB module completes the detection of targets with directions, and outputs the final detection results, including the position and rotation angle of each target.

6. A method for detecting and removing three-phase codes on a production line according to claim 1, characterized in that: The improved YOLO11-cls network includes a backbone network part and a head network part; The backbone network part includes a first Conv module, a second Conv module, a first C3k2 module, a third Conv module, a second C3k2 module, a fourth Conv module, a third C3k2 module, a fifth Conv module, a fourth C3k2 module, an SPPF module, a C2PSA module and a SimAM attention mechanism module which are connected in sequence; The head network part includes a first upsampling module Upsample, a first concatenation module Concat, a fifth C3k2 module, a second upsampling module Upsample, a second concatenation module Concat, a sixth C3k2 module, a sixth Conv module, a third concatenation module Concat, a seventh C3k2 module, a seventh Conv module, a fourth concatenation module Concat, an eighth C3k2 module and a Classify module connected in sequence; The implementation process of the backbone network part is as follows: after the input passes through the first Conv module [64, 3, 2] to extract preliminary features, the features enter the second Conv module [128, 3, 2] for downsampling, and then continue to enter the first C3k2 module [256, False, 0.25] to further extract features; after the feature map passes through the third Conv module [256, 3, 2] for downsampling, the second C3k2 module [512, False, 0.25] is used to process the features on the P3 / 8 scale; and then passes through the fourth After the Conv module [512,3,2] is downsampled to P4 / 16, the third C3k2 module [512,True] is used to perform deeper feature extraction; then it is downsampled to P5 / 32 by the fifth Conv module [1024,3,2] and processed by the fourth C3k2 module [1024,True]; the feature map is then passed through the SPPF module [1024,5] to extract multi-scale features; after passing through the C2PSA module, the SimAM attention mechanism module [1024] is used to focus on key information; The implementation process of the head network part is as follows: the P5 feature map output from the SimAM attention mechanism module is upsampled by 2 times by the first upsampling module Upsample, and then the upsampled P5 feature map is spliced ​​with the P4 feature map by the first concatenation module Concat; the spliced ​​P4 feature map is processed by the fifth C3k2 module [512, False], and the output feature map is upsampled by 2 times by the second upsampling module Upsample to obtain the upsampled P4 feature map, and then the upsampled feature map is spliced ​​with the P3 feature map by the second concatenation module Concat; the spliced ​​P3 feature map is processed by the sixth C3k2 module [256, False], and then downsampled by the sixth Conv module [256, 3, 2] to return to the P4 scale to obtain the downsampled P4 feature map. Figure; the downsampled P4 feature map is concatenated with the P4 feature map processed by the fifth C3k2 module [512, False] through the third concatenation module Concat, and then the features are further extracted through the seventh C3k2 module [512, False], and then downsampled through the seventh Conv module [512, 3, 2] to return to the P5 scale to obtain the P5 feature map after P5 downsampling; the downsampled P5 feature map is concatenated with the feature map of the P5 scale of the backbone network through the fourth concatenation module Concat, and then processed through the eighth C3k2 module [1024, True] to output the final features of the P5 scale; the feature maps of the three scales of P3, P4 and P5 are input into the Classify, Classify classifies the detection targets, and outputs the final detection results, including the categories corresponding to each target.

7. A method for detecting and removing three-phase codes on a production line according to claim 1, characterized in that: The improved YOLO11-obb network and the improved YOLO11-cls network are both accelerated by tensorRT.

8. A detection and rejection system for three-phase codes on a production line, used to implement a detection and rejection method for three-phase codes on a production line as claimed in any one of claims 1 to 7, characterized in that: It includes a conveyor line, a camera, a detection and analysis mechanism and a rejection mechanism. The conveyor line is used to convey products to be detected. The camera and the rejection mechanism are respectively arranged beside or above the conveyor line, and the rejection mechanism is located downstream of the camera. The camera is used to shoot the products to be detected on the conveyor line to obtain images with product three-phase codes, and transmit the images to the detection and analysis mechanism; the detection and analysis mechanism is respectively connected to the camera and the rejection mechanism, and is used to receive the images uploaded by the camera, identify and judge the product three-phase codes on the images, and send the rejection information to the rejection mechanism, and perform statistical analysis on the detection results; the rejection mechanism is used to reject the products corresponding to the unqualified three-phase codes on the conveyor line according to the received instructions; The detection and analysis mechanism includes a model automatic training module, an image receiving and processing module, an improved YOLO11-obb network module, an improved YOLO11-cls network module, an Opencv module, and a data analysis and statistics module; The model automatic training module is used to automatically train the improved YOLO11-obb network and the improved YOLO11-cls network; The image receiving and processing module is used to receive images with product phase III codes uploaded by the camera, and to normalize, scale and splice the images; The improved YOLO11-obb network module is used to identify the position information and rotation angle of each character on the image, and a SimAM attention mechanism is added to it; The Opencv module is used to cut out each character from the image according to the position information and rotation angle of each character, and rotate each character to a normal angle; The improved YOLO11-cls network module is used for batch classification and recognition of all characters in a batch, and a SimAM attention mechanism is added to it; The data analysis and statistics module is used to sort the recognized characters according to their position information, and compare them with standard texts to determine whether they are qualified or not, and to count the total number of tested products, the number of unqualified products, the rejection rate, the types of unqualified products and the number of each type.

9. A detection and rejection system for three-phase codes on a production line according to claim 8, characterized in that: It also includes a control display mechanism, which is connected to the detection and analysis mechanism and is used to display the operation interface of the model automatic training module and the data analysis and statistics module for users to operate, and display the results of automatic labeling and automatic statistics for users to view.

10. A detection and rejection system for three-phase codes on a production line according to claim 8, characterized in that: The model automatic training module includes a picture acquisition unit, a position data set automatic generation unit, a character category data set automatic generation unit, an improved YOLO11-obb network training unit, and an improved YOLO11-cls network training unit; The picture acquisition unit is used to acquire the picture to be trained through a camera; The position data set automatic generation unit includes a SAM visual large model, which is used to automatically identify the position of characters on the picture, automatically mark and extract the characters, and automatically generate a position data set according to the position of the characters; The character category data set automatic generation unit includes a text classification model, which is used to classify and identify the extracted characters, and automatically generate a character category data set according to the identification results; The improved YOLO11-obb network training unit is used to input the data in the position data set into the improved YOLO11-obb network for model training; The improved YOLO11-cls network training unit is used to input data in the character category data set into the improved YOLO11-cls network for model training.

Images