A deep learning-based electric wire lettering printing error detection method

Abstract

The application discloses a kind of based on deep learning's electric wire trace printing error detection method, including the following steps: obtaining the electric wire photo to be detected;Based on pre-constructed and trained deep learning detection model based on attention mechanism, the electric wire photo to be detected is detected, and detection result is output;Wherein, based on the backbone network of deep learning detection model, the characteristic information of the electric wire photo to be detected is extracted;Based on the feature enhancement extraction network of deep learning detection model, the characteristic information is aggregated feature processing, and updated characteristic information is obtained;Based on the prediction network of deep learning detection model, according to updated characteristic information, obtain multiple prediction data;Based on the output network of deep learning detection model, multiple prediction data are carried out dimension conversion and data splicing operation, and detection result is output.The application can achieve high-precision detection, fast detection speed, while deep learning detection model is smaller in size, easy to deploy on edge device.

Description

Technical Field

[0001] This invention relates to a method for detecting printing errors in wire lettering based on deep learning, belonging to the field of target detection technology in computer vision. Background Technology

[0002] Among all electrical wire-related inspections, the detection of printing errors in markings is a crucial type. National standards require that markings on electrical wires and cables be abrasion-resistant and continuous, while maintaining high clarity. Different wire manufacturers have different standards for lettering printing, but generally, some standards are universal, such as clear printing, no oil stains or unclear printing, no missing or illegible prints, and no layout errors or slanted text.

[0003] The traditional method for inspecting the quality of printed text on electrical wires involves having inspectors go to the wire production line, pick up wires, and carefully observe the printed text for missing or incomplete print, whether the wires have been contaminated with oil during production, and whether horizontal lines in the printed text are slanted. This method has significant drawbacks: it is extremely costly in terms of manpower and time. While the printing speed on wires is very fast, inspectors still need considerable time to check whether the text meets standards; moreover, some wires are thin, making the printed text very small and difficult to observe with the naked eye; if the printed text is very similar in color to the wire itself, misjudgments can occur. Prolonged exposure to this work can also put a significant strain on the eyes of inspectors, negatively impacting their physical and mental health. Furthermore, because printed characters on wires tend to repeat periodically, inspectors need to statistically analyze the printing of all characters on every wire before they can determine whether the text printing on the wire is up to standard.

[0004] There are many methods for applying computer vision technology to text recognition. The patent "An Image Text Recognition System (CN102136064A, 2011.07.27)" uses computer vision technology to extract text regions from images. After character segmentation, background removal, and feature comparison, it can recognize the text in the image. However, applying this patent's method to wire marking detection will not achieve the desired results. The reasons are as follows: First, the text on wires is small and densely distributed, causing problems in the character segmentation module of the patent's technology, making it impossible to accurately segment each character. Second, the printing speed of wires on an assembly line is very fast, while the method used in this patent is too slow to perform the detection in time. Finally, what needs to be detected on wires is printing errors. If printing errors occur, we only need to know which characters are incorrect and need to be reprinted. Therefore, in the wire and cable industry, there is a need for a new method to detect handwriting that can achieve high-precision detection, detect at a speed that can keep up with the speed of wire and cable production lines, and has a small deep learning detection model that is easy to deploy on edge devices.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for detecting printing errors in electrical wires based on deep learning. This method can achieve high-precision detection, detect at a speed that can keep up with the speed of electrical wire production lines, and the deep learning detection model is small in size and easy to deploy on edge devices.

[0007] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0008] This invention discloses a method for detecting printing errors in wire lettering based on deep learning, comprising the following steps:

[0009] Obtain a photograph of the wire to be inspected;

[0010] Based on a pre-built and trained deep learning detection model based on an attention mechanism, the image of the wire to be detected is detected, and the detection result is output.

[0011] The detection method of the deep learning detection model is as follows:

[0012] Based on the backbone network of the deep learning detection model, feature information of the wire photograph to be detected is extracted;

[0013] Based on the feature enhancement extraction network of the deep learning detection model, the feature information is aggregated and processed to obtain updated feature information;

[0014] Based on the prediction network of the deep learning detection model, multiple prediction data are obtained according to the updated feature information;

[0015] Based on the output network of the deep learning detection model, dimensionality transformation and data concatenation operations are performed on the multiple prediction data to output the detection results.

[0016] Furthermore, the training method for the deep learning detection model is as follows:

[0017] Obtain the training set for model training;

[0018] The training set is input into a pre-built deep learning detection model based on attention mechanism for iterative training, and the model parameters are updated until the accuracy of the character category of the detection result output by the pre-built deep learning detection model based on attention mechanism reaches a preset accuracy threshold, thus obtaining a pre-built and trained deep learning detection model based on attention mechanism.

[0019] Furthermore, obtaining the training set for model training includes the following steps:

[0020] Obtain a dataset for model training, wherein the dataset includes multiple sets of wire photos of different types, and each set of wire photos of different types includes multiple photos of wires with defective character printing and corresponding multiple photos of wires with acceptable character printing.

[0021] For each photograph of a wire in the dataset, the characters of the wire are labeled and categorized to obtain the label box and category result for each character; the category result includes characters that are not printed properly and characters that are printed properly.

[0022] Based on the labeled frame, the wire photos corresponding to the labeled frame are cropped to obtain multiple cropped wire photos;

[0023] Based on the cropped wire photos and the corresponding character category results, a training set for model training is obtained.

[0024] Furthermore, the backbone network of the deep learning detection model includes an input module, an image extraction module, a first combination module, a first residual module, a second combination module, a second residual module, a third combination module, a third residual module, a spatial pyramid pooling module, and a first encoding module connected in sequence.

[0025] Furthermore, the feature enhancement extraction network of the deep learning detection model includes a fourth combination module, a first upsampling module, a first concatenation module, a fourth residual module, a fifth combination module, a second upsampling module, a second concatenation module, a fifth residual module, a sixth combination module, a third upsampling module, a third concatenation module, a second encoding module, a seventh combination module, a fourth concatenation module, a third encoding module, an eighth combination module, a fifth concatenation module, and a fourth encoding module;

[0026] The fourth combination module is connected to the output of the first encoding module, and the fourth encoding module is connected to one output of the fourth combination module; the first splicing module is connected to one output of the third residual module; the second splicing module is connected to one output of the second residual module; the third splicing module is connected to one output of the first residual module; the fourth splicing module is connected to one output of the sixth combination module; and the fifth splicing module is connected to one output of the fifth combination module.

[0027] Furthermore, the prediction network of the deep learning detection model includes a first convolutional module, a first information processing module, a second convolutional module, a second information processing module, a third convolutional module, and a third information processing module.

[0028] The input of the first convolution module is connected to the second encoding module, and the output of the first convolution module is connected to the first information processing module; the input of the second convolution module is connected to the third encoding module, and the output of the second convolution module is connected to the second information processing module; the input of the third convolution module is connected to the fourth encoding module, and the output of the third convolution module is connected to the third information processing module.

[0029] Furthermore, the output network of the deep learning detection model includes a first dimension transformation module, a second dimension transformation module, a third dimension transformation module, a sixth concatenation module, and an output module;

[0030] The input of the first dimension conversion module is connected to the first information processing module, and the output of the first dimension conversion module is connected to the sixth splicing module; the input of the second dimension conversion module is connected to the second information processing module, and the output of the second dimension conversion module is connected to the sixth splicing module; the input of the third dimension conversion module is connected to the third information processing module, and the output of the third dimension conversion module is connected to the sixth splicing module; the output of the sixth splicing module is connected to the output module.

[0031] Furthermore, the first encoding module, and / or the second encoding module, and / or the third encoding module, and / or the fourth encoding module include an input submodule, a first normalization submodule, an attention mechanism submodule, a first addition submodule, a second normalization submodule, an information processing submodule, a second addition submodule, and an output submodule connected in sequence.

[0032] The first addition submodule is connected to one output of the input submodule; the second addition submodule is connected to one output of the second normalization submodule.

[0033] Furthermore, the attention mechanism submodule includes a first input unit, a K data unit, a Q data unit, a V data unit, a first multiplication unit, a Softmax data calculation unit, a second multiplication unit, a first Linear data calculation unit, a first regularization unit, and a first output unit;

[0034] Wherein; the output of the first input unit is connected to the K data unit, the Q data unit and the V data unit respectively; the outputs of the K data unit and the Q data unit are connected to the first multiplication unit respectively; the first multiplication unit is connected to one input of the second multiplication unit through the Softmax data calculation unit, and the V data unit is connected to the other input of the second multiplication unit; the second multiplication unit is connected to the first output unit in sequence through the first Linear data calculation unit and the first regularization unit.

[0035] Furthermore, the first information processing module, and / or the second information processing module, and / or the third information processing module, and / or the information processing submodule include a second input unit, a second linear data calculation unit, an activation unit, a second regularization unit, a third linear data calculation unit, a third regularization unit, and a second output unit connected in sequence.

[0036] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0037] This invention innovatively applies target detection technology to the field of wire lettering detection, solving the problem that the wire lettering detection industry still relies on manual inspection. At the same time, it improves upon traditional deep learning detection models by introducing an attention mechanism, enabling them to achieve higher accuracy in wire lettering detection. This high-precision model can achieve even greater accuracy in determining whether wire lettering printing is qualified. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of the deep learning detection model based on the attention mechanism provided in the embodiment;

[0039] Figure 2This is a schematic diagram of the structure of the encoding module provided in the embodiment;

[0040] Figure 3 This is a schematic diagram of the attention mechanism submodule provided in the embodiment;

[0041] Figure 4 This is a schematic diagram of the structure of the information processing module or information processing submodule provided in the embodiment;

[0042] Figure 5 This is the workflow for detecting lettering on electrical wires provided in the embodiment;

[0043] Figure 6 It is a positive sample of the first type of data sample image provided in the embodiment;

[0044] Figure 7 It is a negative sample of the first type of data sample image provided in the embodiment;

[0045] Figure 8 It is a positive sample of the second type of data sample image provided in the embodiment;

[0046] Figure 9 It is a negative sample of the second type of data sample image provided in the embodiment;

[0047] Figure 10 It is a positive sample of the third type of data sample image provided in the embodiment;

[0048] Figure 11 It is a negative sample of the third type of data sample image provided in the embodiment;

[0049] Figure 12 It is a positive sample of the fourth type of data sample image provided in the embodiment;

[0050] Figure 13 It is a negative sample of the fourth type of data sample image provided in the embodiment;

[0051] Figure 14 This is an example diagram of wire marking detection provided in the embodiment. Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0053] This embodiment discloses a method for detecting printing errors in wire lettering based on deep learning, including the following steps:

[0054] Obtain a photograph of the wire to be inspected;

[0055] Based on a pre-built and trained deep learning detection model based on an attention mechanism, the system detects the wire images to be detected and outputs the detection results.

[0056] The detection methods of the deep learning detection model are as follows:

[0057] Based on the backbone network of the deep learning detection model, feature information of the photos of the wires to be detected is extracted.

[0058] A feature enhancement extraction network based on a deep learning detection model is used to aggregate feature information and obtain updated feature information.

[0059] A prediction network based on a deep learning detection model obtains multiple prediction data based on the updated feature information.

[0060] The output network of the deep learning detection model performs dimensionality transformation and data concatenation operations on multiple prediction data to output the detection results.

[0061] The technical concept of this invention is as follows: because the printed text on electrical wires is relatively dense and the characters are small, some of the printed text has low recognition accuracy, and the color is very similar to that of the electrical wires, conventional text recognition technology cannot be perfectly adapted to electrical wire text detection. Therefore, this invention innovatively proposes to apply object detection technology to electrical wire text detection. At the same time, since general deep learning models cannot achieve high recognition accuracy when directly used to train electrical wire text detection, this invention adopts an optimized deep learning detection model based on an attention mechanism, which can achieve a higher accuracy in electrical wire text detection. The high-precision model can have a higher accuracy in judging whether the printed electrical wire text is qualified.

[0062] The specific steps are as follows:

[0063] Step 1: Model training.

[0064] The model training part requires a relatively comprehensive dataset. In this example, an industrial camera was used to capture a dataset of wire markings. This dataset contains a total of 715 images, covering four common types of wire printing defects that may occur in production lines.

[0065] like Figure 6 and Figure 7 As shown, the first type of data sample image is compared with positive and negative samples. In this data sample, the printing error of the negative sample is that it is blurry and the printed characters appear as dots and cannot be identified.

[0066] like Figure 8 and Figure 9As shown, the second type of data sample image compares positive and negative samples. The printing error in the negative sample is that oil stains appeared during the printing process, which had a certain impact on the printing quality.

[0067] like Figure 10 and Figure 11 As shown, the third type of data sample image compares positive and negative samples. The printing error in the negative sample is printing offset. When printed correctly, each character will have a shorter horizontal line above and below it, while the negative sample will have missing horizontal lines.

[0068] like Figure 12 and Figure 13 As shown, the positive and negative samples of the fourth type of data sample image are compared. The printing error of the negative sample in this data sample is that the printing is missing. Only half of the printed characters in the negative sample are visible, and the other half are not printed.

[0069] The four types of data sample images formed four different types of comparisons. Each type of wire photo consisted of about 160 images. The number of wire photos with defective character printing was equal to the number of wire photos with qualified character printing, about 80 images each.

[0070] After obtaining the photos of the wires, the images need to be labeled. Using a relevant labeling tool (such as labelImg), each character on the wire is labeled, resulting in a bounding box and category for each character. Based on the detection criteria, the characters are divided into two categories: characters with printing defects and characters with printing defects. It should be noted that both the wire photos with and without printing defects in the dataset were manually selected; the characters with and without printing defects were also manually selected. This is to facilitate subsequent model training until the accuracy of the character categories output by the pre-built attention-based deep learning detection model reaches a preset accuracy threshold, thus obtaining a trained model.

[0071] Because the original images captured by industrial cameras are too large (4024*3036 pixels), directly using them as training data for the model would result in excessively long training times and poor training performance for deep learning models. Therefore, before splitting the dataset, the images need to be cropped to approximately 640*640 pixels based on the annotation information. The cropping method is as follows:

[0072] 1. Set the configuration file for annotation and the image reading path.

[0073] 2. Read the configuration file and input the position information of all the labeled boxes in the image (top left and bottom right corners) into four lists (x-axis coordinate of top left corner, y-axis coordinate of top left corner, x-axis coordinate of bottom right corner, and y-axis coordinate of bottom right corner).

[0074] 3. Sort the position information of the annotation boxes according to the sorting rule: sort by the x-axis coordinate of the top left corner point from smallest to largest.

[0075] 4. Find the annotation box with the highest top boundary and the lowest bottom boundary among all annotation boxes, calculate the difference between the two, and pad the height of the cropped image to 640 pixels. If the padded position exceeds the image boundary, then padded to the image boundary position.

[0076] 5. Crop the image from the left side, starting at the x-coordinate of the top left corner, cropping a width of 640 pixels and a height equal to the coordinates obtained in step 4.

[0077] 6. When the width of the last captured image is less than 640 pixels, crop it to the rightmost edge.

[0078] After the above cropping process, the number of images becomes about 3800, and on average, each image can be cropped into 6 images of 640*640 pixels each.

[0079] Based on the cropped wire photos and the corresponding character category results, the labeled cropped wire photos are divided into training and validation sets according to a certain ratio.

[0080] After acquiring the training set, the model training process begins. Through comparative experiments with other deep learning models, this invention ultimately selected the YOLOv5 model. As a one-stage detection model, YOLOv5 offers both high detection speed and relatively high accuracy. Furthermore, the YOLOv5 model is small in size, enabling deployment on various edge devices.

[0081] As mentioned above, general deep learning models suffer from low accuracy in detecting handwriting on electrical wires. This invention employs an optimized model, referencing the Attention mechanism of the Transformer model in Natural Language Processing (NLP). The model framework of the YOLOv5 model has been modified, and the modified YOLOv5 model structure, namely the attention-based deep learning detection model provided in this embodiment, is as follows: Figure 1 As shown, the shadow module is the core setting of this model.

[0082] It should be noted that the deep learning detection model based on the attention mechanism in this embodiment is a further modification of the YOLOv5 convolutional neural network. The backbone network in the deep learning detection model is equivalent to the backbone of the YOLOv5 convolutional neural network; the feature enhancement extraction network is equivalent to the neck of the YOLOv5 convolutional neural network; the prediction network is equivalent to the head of the YOLOv5 convolutional neural network; and the output network is equivalent to the output of the YOLOv5 convolutional neural network. The backbone network of the deep learning detection model includes, in sequence, an input module, an image extraction module, a first combination module, a first residual module, a second combination module, a second residual module, a third combination module, a third residual module, a spatial pyramid pooling module, and a first encoding module.

[0083] Specifically, the backbone network is used to extract feature information from the input image. The image extraction module is used to downsample the image without losing image feature information. The first combination module, and / or the second combination module, and / or the third combination module are combinations of three operations: convolution (Conv), normalization (BN), and Leaky ReLU type activation operation. The first residual module CSP1_1*3, where the first 1 after CSP indicates that the module contains residual components, and the second 1 indicates that one residual unit is used; the second residual module is similar to the third. The Spatial Pyramid Pooling module, SPP, is responsible for pooling image features and generating a fixed-length output. The first encoding module is used to better extract and analyze image feature values.

[0084] The feature enhancement extraction network of the deep learning detection model includes a fourth combination module, a first upsampling module, a first concatenation module, a fourth residual module, a fifth combination module, a second upsampling module, a second concatenation module, a fifth residual module, a sixth combination module, a third upsampling module, a third concatenation module, a second encoding module, a seventh combination module, a fourth concatenation module, a third encoding module, an eighth combination module, a fifth concatenation module, and a fourth encoding module;

[0085] Specifically, the fourth combination module is connected to the output of the first encoding module, and the fourth encoding module is connected to one output of the fourth combination module; the first splicing module is connected to one output of the third residual module; the second splicing module is connected to one output of the second residual module; the third splicing module is connected to one output of the first residual module; the fourth splicing module is connected to one output of the sixth combination module; and the fifth splicing module is connected to one output of the fifth combination module.

[0086] The feature enhancement extraction network aggregates image information from different depths in the backbone network. The fourth, fifth, sixth, seventh, and eighth combination modules are identical to those in the backbone network, consisting of three operations: convolution (Conv), batch normalization (BN), and Leaky ReLU activation. The second, third, and fourth encoding modules function the same as the first encoding module in the backbone network. The first, second, and third upsampling modules amplify the feature information in the image. The fourth and fifth residual modules, CSP2_1*3, have a 2 indicating that the residual components in this module have been replaced with CBLs, and a 1 indicating that there are 2*1 CBL modules in this module. The first, second, third, fourth, and fifth concatenation modules concatenate multiple inputs into a single output.

[0087] It should be noted that the backbone network is used to extract feature information from the input image, mainly some basic feature information; the feature enhancement extraction network is used to perform feature aggregation processing based on the feature information extracted by the backbone network, aggregating image information at different depths, and further extracting feature information to obtain updated feature information.

[0088] The prediction network of the deep learning detection model includes a first convolutional module, a first information processing module, a second convolutional module, a second information processing module, a third convolutional module, and a third information processing module.

[0089] The input of the first convolution module is connected to the second encoding module, and the output of the first convolution module is connected to the first information processing module; the input of the second convolution module is connected to the third encoding module, and the output of the second convolution module is connected to the second information processing module; the input of the third convolution module is connected to the fourth encoding module, and the output of the third convolution module is connected to the third information processing module.

[0090] Specifically, the prediction network is used to predict the detection results of the input image. The first, second, and third convolutional modules are used to further extract features from the input information, extracting information features from three different depths of the feature enhancement extraction network. The first, second, and third information processing modules are used to decode the corresponding feature information.

[0091] The output network of the deep learning detection model includes a first-dimensional transformation module, a second-dimensional transformation module, a third-dimensional transformation module, a sixth-dimensional concatenation module, and an output module;

[0092] The input of the first dimension conversion module is connected to the first information processing module, and the output of the first dimension conversion module is connected to the sixth splicing module; the input of the second dimension conversion module is connected to the second information processing module, and the output of the second dimension conversion module is connected to the sixth splicing module; the input of the third dimension conversion module is connected to the third information processing module, and the output of the third dimension conversion module is connected to the sixth splicing module; the output of the sixth splicing module is connected to the output module.

[0093] Specifically, the output network is used to perform dimensionality transformation and data concatenation operations on the three outputs of the prediction network, ultimately obtaining the prediction result of the input image.

[0094] like Figure 2 As shown, the first encoding module, and / or the second encoding module, and / or the third encoding module, and / or the fourth encoding module are designed to better extract and analyze image feature values. They include an input submodule, a first normalization submodule, an attention mechanism submodule, a first addition submodule, a second normalization submodule, an information processing submodule, a second addition submodule, and an output submodule connected in sequence.

[0095] The first addition submodule is connected to one output of the input submodule; the second addition submodule is connected to one output of the second normalization submodule.

[0096] Specifically, the first or second normalization submodule is used to normalize the data; the attention mechanism submodule is used to mine the representational potential of the input image features; and the information processing submodule is used to decode the feature information. The specific process is as follows: First, the input data is normalized. Then, the attention mechanism submodule continues to extract image feature information. After the processed data is normalized again, it is decoded by the information processing submodule to obtain the output information.

[0097] like Figure 3 As shown, the attention mechanism submodule includes a first input unit, a K data unit, a Q data unit, a V data unit, a first multiplication unit, a Softmax data calculation unit, a second multiplication unit, a first Linear data calculation unit, a first regularization unit, and a first output unit;

[0098] Wherein; the output of the first input unit is connected to the K data unit, the Q data unit and the V data unit respectively; the outputs of the K data unit and the Q data unit are connected to the first multiplication unit respectively; the first multiplication unit is connected to one input of the second multiplication unit through the Softmax data calculation unit, and the V data unit is connected to the other input of the second multiplication unit; the second multiplication unit is connected to the first output unit in sequence through the first Linear data calculation unit and the first regularization unit.

[0099] Specifically, the attention mechanism submodule is used to mine the representational potential of input image features. The Softmax data computation unit is used to change the distribution of data information; the first Linear data computation unit is used to perform linear operations (fully connected) on the data; the first regularization unit is used to regularize the data and discard some unnecessary data. The workflow of the attention mechanism submodule is as follows: First, the input data is processed using three 1*1 convolutional kernels (Conv operation) to obtain three sets of data K, Q, and V. Then, K and Q are multiplied, and the result is processed by Softmax. Then, it is multiplied by V. The result is then processed by Linear operation and Dropout regularization operation, and finally the output of the attention mechanism submodule is obtained.

[0100] like Figure 4 As shown, the first information processing module, and / or the second information processing module, and / or the third information processing module, and / or the information processing sub-module include a second input unit, a second linear data calculation unit, an activation unit, a second regularization unit, a third linear data calculation unit, a third regularization unit, and a second output unit connected in sequence.

[0101] Specifically, the information processing module or information processing submodule is used to decode the feature information. The second and third Linear data calculation units have the same function as the first Linear data calculation unit mentioned above, and the second and third regularization units have the same function as the first regularization unit mentioned above. The activation unit GELU is used to change the data distribution of the input data. The specific process is as follows: First, the input data is subjected to a Linear operation; then, the data distribution is changed through the activation unit GELU; next, a portion of the data information is discarded using the second regularization unit, and a Linear operation is performed again, followed by another portion of the data information being discarded using the third regularization unit, finally obtaining the output result.

[0102] Thanks to the introduction of the attention mechanism from Transformer, the YOLOv5 model can achieve higher accuracy in small object detection.

[0103] The preprocessed training set is input into the improved YOLOv5 network model for training. The parameters are adjusted according to the training results, and the model is iterated. When the accuracy of the model in testing a single character category reaches more than 97%, the deep learning detection model based on the attention mechanism can be considered to have completed training. At this time, the deep learning detection model based on the attention mechanism can be used for subsequent deployment.

[0104] Step 2: Model Deployment.

[0105] The model deployment part requires installing the necessary environment for running deep learning models on edge devices, including Torch, OpenCV, NumPy, etc. Simultaneously, interfaces for calling the model and calling the camera are written in a programming language (e.g., Python), and the front-end and back-end of this invention are fully implemented using the Python Django framework. After connecting all parts, the complete form of this invention is achieved. Inspection personnel can then perform operations on the front-end to detect characters on the wire printing production line.

[0106] Step 2: Model usage.

[0107] Obtain a photograph of the wire to be inspected;

[0108] Based on a pre-built and trained deep learning detection model with an attention mechanism, the system detects the wire images to be detected and outputs the detection results.

[0109] Actual operation as follows Figure 5 As shown, during the wire marking inspection, staff control a camera via a designated button on the front-end webpage to capture images of the printed markings on the wire printing production line. Because the industrial photos of the production line are too large, the images are cropped to an average size based on the actual situation. The cropped images are then input into a deployed deep learning detection model based on an attention mechanism through a relevant interface for inspection. After the model completes its inspection, it returns the detection results of all characters on the current wire image to the front-end webpage. The wire marking inspection results are as follows: Figure 14 As shown. Simultaneously, the system backend, based on the pass / fail status of the characters detected on the wire, uses an algorithm to determine whether the characters printed on the wire meet production standards, and returns the results to the front-end webpage. On the front-end webpage, staff can see the image of the wire characters captured by the camera, the detection results of all characters in the wire image, and the overall detection results for the wire. In the detection result image, 'n' is an abbreviation for 'negative,' representing that the framed area of ​​the image was identified as a non-compliant character in the wire after detection; 'p' is an abbreviation for 'positive,' representing that the framed area of ​​the image was identified as a compliant character in the wire after detection; the decimal following 'n' or 'p' represents the probability of the model's prediction result, indicating the probability that the text at that location is printed compliantly (or non-compliantly). The numerical value is between 0 and 1.

[0110] Because text is printed in multiple places on a single wire, and the content of these texts is identical, after multiple tests and analyses, it was found that if the total number of characters judged as unacceptable by the model in a single photograph is less than 5% of all characters, the text printing on the wire can be considered acceptable. This concludes one wire text printing acceptance test.

[0111] According to tests and statistics, the accuracy rate of detecting individual characters in wire markings using this invention is as high as 98%, the overall pass rate for detecting markings on a single wire is over 96%, and the time to detect a single image is approximately 120ms, which basically meets the detection speed requirements of the wire printing industry.

[0112] In summary, the text recognition method mentioned in the background section utilizes image recognition technology from the field of computer vision. However, its image reading and processing speed is relatively slow and cannot meet the speed requirements of industrial production lines in the wire and cable industry. This embodiment, on the other hand, is based on deep learning detection technology, which can detect an image in approximately 100 milliseconds on a high-performance edge device. Therefore, this invention can be directly applied to industrial production lines to detect printing problems in real time on wire and cable printing lines.

[0113] When performing detection, deep learning models return the image's category information and the probability (confidence) that the image belongs to that category. Therefore, the results can be directly plotted on the detected image. This invention, after calling a deep learning model to detect lettering on electrical wires, displays the image results on the front-end page, making the detection results more intuitive.

[0114] The detection method used in this invention is simple to operate. After the hardware is deployed, the testing personnel only need to click the buttons for shooting, testing and other related functions on the designated front-end page to realize the detection of lettering on the wires.

[0115] Because the printed text on electrical wires is dense and the characters are small, some of the printed text has low recognition accuracy, and the color is very similar to that of the wires. Therefore, if a general deep learning model is directly used to train on the wire text recognition, it cannot achieve a high recognition accuracy. This invention optimizes the deep learning model used, changing some of its structure, so that it can achieve a higher accuracy in detecting wire text. The high-precision model can have a higher accuracy in judging whether the printed text on electrical wires is qualified.

[0116] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting printing errors in electrical wire lettering based on deep learning, characterized in that, Includes the following steps: Obtain a photograph of the wire to be inspected; Based on a pre-built and trained deep learning detection model based on an attention mechanism, the image of the wire to be detected is detected, and the detection result is output. The detection method of the deep learning detection model is as follows: Based on the backbone network of the deep learning detection model, feature information of the wire photograph to be detected is extracted; Based on the feature enhancement extraction network of the deep learning detection model, the feature information is aggregated and processed to obtain updated feature information; Based on the prediction network of the deep learning detection model, multiple prediction data are obtained according to the updated feature information; Based on the output network of the deep learning detection model, dimensionality transformation and data concatenation operations are performed on the multiple prediction data to output the detection results. The backbone network of the deep learning detection model includes an input module, an image extraction module, a first combination module, a first residual module, a second combination module, a second residual module, a third combination module, a third combination module, a spatial pyramid pooling module, and a first encoding module, connected in sequence. The feature enhancement extraction network of the deep learning detection model includes a fourth combination module, a first upsampling module, a first splicing module, a fourth residual module, a fifth combination module, a second upsampling module, a second splicing module, a fifth residual module, a sixth combination module, a third upsampling module, a third splicing module, a second encoding module, a seventh combination module, a fourth splicing module, a third encoding module, an eighth combination module, a fifth splicing module, and a fourth encoding module; Wherein, the fourth combining module is connected to the output of the first encoding module, and the fourth encoding module is connected to one output of the fourth combining module; the first splicing module is connected to one output of the third residual module; the second splicing module is connected to one output of the second residual module; the third splicing module is connected to one output of the first residual module; the fourth splicing module is connected to one output of the sixth combining module; and the fifth splicing module is connected to one output of the fifth combining module. The prediction network of the deep learning detection model includes a first convolutional module, a first information processing module, a second convolutional module, a second information processing module, a third convolutional module, and a third information processing module. Wherein, the input of the first convolution module is connected to the second encoding module, and the output of the first convolution module is connected to the first information processing module; the input of the second convolution module is connected to the third encoding module, and the output of the second convolution module is connected to the second information processing module; the input of the third convolution module is connected to the fourth encoding module, and the output of the third convolution module is connected to the third information processing module. The output network of the deep learning detection model includes a first-dimensional transformation module, a second-dimensional transformation module, a third-dimensional transformation module, a sixth-dimensional concatenation module, and an output module; The input of the first dimension conversion module is connected to the first information processing module, and the output of the first dimension conversion module is connected to the sixth splicing module; the input of the second dimension conversion module is connected to the second information processing module, and the output of the second dimension conversion module is connected to the sixth splicing module; the input of the third dimension conversion module is connected to the third information processing module, and the output of the third dimension conversion module is connected to the sixth splicing module; the output of the sixth splicing module is connected to the output module.

2. The method for detecting printing errors in electrical wire lettering based on deep learning according to claim 1, characterized in that, The training method for the deep learning detection model is as follows: Obtain the training set for model training; The training set is input into a pre-built deep learning detection model based on attention mechanism for iterative training, and the model parameters are updated until the accuracy of the character category of the detection result output by the pre-built deep learning detection model based on attention mechanism reaches a preset accuracy threshold, thus obtaining a pre-built and trained deep learning detection model based on attention mechanism.

3. The method for detecting printing errors in electrical wire lettering based on deep learning according to claim 2, characterized in that, The process of obtaining the training set for model training includes the following steps: Obtain a dataset for model training, wherein the dataset includes multiple sets of wire photos of different types, and each set of wire photos of different types includes multiple photos of wires with defective character printing and corresponding multiple photos of wires with acceptable character printing. For each photograph of a wire in the dataset, the characters of the wire are labeled and categorized to obtain the label box and category result for each character; the category result includes characters that are not printed properly and characters that are printed properly. Based on the labeled frame, the wire photos corresponding to the labeled frame are cropped to obtain multiple cropped wire photos; Based on the cropped wire photos and the corresponding character category results, a training set for model training is obtained.

4. The method for detecting printing errors in electrical wire lettering based on deep learning according to claim 1, characterized in that, The first encoding module, and / or the second encoding module, and / or the third encoding module, and / or the fourth encoding module include an input submodule, a first normalization submodule, an attention mechanism submodule, a first addition submodule, a second normalization submodule, an information processing submodule, a second addition submodule, and an output submodule connected in sequence. The first addition submodule is connected to one output of the input submodule; the second addition submodule is connected to one output of the second normalization submodule.

5. The method for detecting printing errors in electrical wire lettering based on deep learning according to claim 4, characterized in that, The attention mechanism submodule includes a first input unit, a K data unit, a Q data unit, a V data unit, a first multiplication unit, a Softmax data calculation unit, a second multiplication unit, a first Linear data calculation unit, a first regularization unit, and a first output unit; in; The output of the first input unit is connected to the K data unit, the Q data unit, and the V data unit, respectively; the outputs of the K data unit and the Q data unit are connected to the first multiplication unit, respectively; the first multiplication unit is connected to one input of the second multiplication unit through the Softmax data calculation unit, and the V data unit is connected to the other input of the second multiplication unit; the second multiplication unit is connected to the first output unit in sequence through the first Linear data calculation unit and the first regularization unit.

6. The method for detecting printing errors in electrical wire lettering based on deep learning according to claim 5, characterized in that, The first information processing module, and / or the second information processing module, and / or the third information processing module, and / or the information processing submodule include a second input unit, a second linear data calculation unit, an activation unit, a second regularization unit, a third linear data calculation unit, a third regularization unit, and a second output unit connected in sequence.

Images