Steel pipe multi-line handwritten character detection method and device, electronic equipment and storage medium

Abstract

The application provides a steel pipe multi-line handwritten character detection method and device, electronic equipment and storage medium, relates to the technical field of steel pipe handwritten character detection method, and comprises the following steps: extracting a multi-line handwritten character region on a steel pipe image to obtain a character region image containing multi-line handwritten characters; converting the character region image into a gray scale image; equally cutting the gray scale image along the vertical direction to obtain a plurality of vertical segment images; obtaining a corresponding projection curve for each vertical segment image by using a horizontal projection method, and determining a character line region of each line of characters according to the projection curve of the vertical segment image; extracting a single character region in the character line region; and performing character recognition on the single character region on the steel pipe image by using a preset recognition model to obtain a recognition result. The application can be applied to the recognition of handwritten characters on a steel pipe in an actual production process, and can efficiently and accurately recognize handwritten characters on the steel pipe.

Description

Technical Field

[0001] This application relates to the technical field of handwritten character detection methods for steel pipes, and in particular to a method, apparatus, electronic device, and storage medium for detecting multi-line handwritten characters on steel pipes. Background Technology

[0002] With the increasingly widespread application of steel pipes, the rapid and efficient identification of steel pipe markings has become a crucial breakthrough for many factories to improve efficiency. Identifying steel pipe markings is essential and time-consuming at multiple stages of the steel pipe production process, requiring a large number of manual laborers. Only after correct identification can the next process begin, which not only significantly increases production costs but also carries the risk of errors due to manual identification, potentially leading to major production accidents and substantial economic losses. Furthermore, manual identification requires operators to insert their heads into the steel pipes, posing significant safety risks. Therefore, replacing manual steel pipe marking with automated machines has become an inevitable trend.

[0003] Specifically, the production process of steel pipes is as follows: Figure 1 As shown, the above process includes 20 positions where workers need to copy and register the following steps: steel plate feeding inspection, automatic ultrasonic flaw detection of steel plates, edge milling, pre-bending, steel pipe forming, pre-welding of joints, pre-weld repair, internal welding, external welding, manual ultrasonic flaw detection of welds before diameter expansion, X-ray industrial television inspection before diameter expansion, repair welding, mechanical diameter expansion, hydrostatic testing, pipe end weld grinding, pipe end chamfering, automatic ultrasonic flaw detection of welds after diameter expansion and hydrostatic testing, X-ray radiography of pipe ends after diameter expansion and hydrostatic testing, finished product inspection, and warehousing inspection. The steel plate feeding inspection position handwrites important information such as pipe number, specifications, and material on the steel plate. Subsequent positions then manually copy the pipe number, specifications, and material information according to the handwritten markings.

[0004] The following difficulties exist in identifying handwritten characters on steel pipes: 1. The handwritten characters are often irregular and difficult to recognize. 2. Before writing pipe numbers and other characters, the steel plate surface needs to be polished. After polishing, the surface is very shiny, and the handwritten markings will be difficult to read due to glare. 3. Finished steel pipes are made by bending a steel plate several times in the forming process. If there is iron oxide scale on the steel plate, it will fall off during the bending process, causing some of the handwritten markings to fall off as well, making them difficult to read. 4. The steel pipes are difficult to read due to oil stains on the inner wall after mechanical diameter expansion and hydrostatic testing. Therefore, there are many difficulties in identifying handwritten characters on steel pipes in actual production processes. This application aims to provide a method for detecting multi-line handwritten characters on steel pipes, so as to efficiently and accurately identify handwritten characters on steel pipes. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a method, device, electronic device and storage medium for detecting multi-line handwritten characters on steel pipes, in order to address the above-mentioned shortcomings of the prior art.

[0006] A method for detecting multi-line handwritten characters on steel pipes, comprising:

[0007] Extract the multi-line handwritten character region from the steel pipe image to obtain a character region image containing multiple lines of handwritten characters;

[0008] Convert the character region image into a grayscale image;

[0009] The grayscale image is divided into multiple vertical segments at equal intervals along the vertical direction.

[0010] For each vertical segment image, a horizontal projection method is used to obtain the corresponding projection curve, and the character row area of ​​each row of characters is determined according to the projection curve of the vertical segment image;

[0011] Extract a single character region from a character line region;

[0012] A preset recognition model is used to perform character recognition on a single character region on the steel pipe image to obtain the recognition result.

[0013] In an improved technical solution, determining the character row region of each row of characters based on the projection curve of the vertical segment image includes:

[0014] Determine the peak points of each line of characters in the projection curve;

[0015] The peak points of each row of characters in the multiple vertical segment images are matched to obtain the character row region of each row of characters.

[0016] In an improved technical solution, the step of mapping the peak points of each row of characters in the plurality of vertical segment images to obtain the character row region of each row of characters includes:

[0017] The peak points of each row of characters in multiple vertical image segments are fitted to obtain several peak fitting lines;

[0018] The middle dividing line at the maximum width position between adjacent peak fitting lines is taken as the dividing line of adjacent character line regions, and each peak fitting line corresponds to a character line region; the upper edge line of the uppermost character line region is generated above the uppermost peak fitting line, and the distance between the upper edge line and the adjacent dividing line below is equal to the width at the maximum width position between the two uppermost peak fitting lines; the lower edge line of the lowermost character line region is generated below the lowermost peak fitting line, and the distance between the lower edge line and the adjacent dividing line above is equal to the width at the maximum width position between the two lowermost peak fitting lines.

[0019] The character line area of ​​each line of characters is determined by the dividing line between adjacent character line areas, the upper edge of the uppermost character line area, and the lower edge of the lowermost character line area.

[0020] In an improved technical solution, extracting a single character region from the character line region includes:

[0021] Convert the images of each character line region to the LAB color space and extract the L channel image;

[0022] The L-channel image is subjected to median filtering.

[0023] Extract the edge features of characters in the L-channel image to obtain an edge feature image;

[0024] Morphological dilation is performed on the edge feature image, and then filling is performed to obtain the filled character image, and small white areas caused by noise are removed;

[0025] Connected component extraction is performed on the filled character image to obtain segmented individual character regions.

[0026] In an improved technical solution, the step of extracting edge features of characters in the L-channel image to obtain an edge feature image includes:

[0027] The edge features in the L-channel image are extracted using the Canny edge operator to form the first image;

[0028] All non-black pixels in the L-channel image are set to white to form a binary image, which is denoted as the second image.

[0029] Perform morphological erosion on the second image to obtain the third image;

[0030] The intersection of the first and third images is used to obtain the fourth image, which is then used as the edge feature image.

[0031] In an improved technical solution, the recognition model is an improved LeNet-5 model; the improved LeNet-5 model adds a self-attention mechanism to the LeNet-5 model, changes the last activation function to use the h-swish activation layer of MobileNetV3, and adds a Dropout layer.

[0032] In an improved technical solution, the extraction of the multi-line handwritten character region on the steel pipe image specifically involves using the YOLOv5 object detection algorithm to extract the multi-line handwritten character region on the steel pipe image.

[0033] On the other hand, this application also provides a steel pipe multi-line handwritten character detection device, comprising:

[0034] The image extraction module is used to extract the multi-line handwritten character region on the steel pipe image to obtain a character region image containing multiple lines of handwritten characters;

[0035] The conversion module is used to convert the character region image into a grayscale image;

[0036] The segmentation module is used to segment the grayscale image vertically at equal intervals to obtain multiple vertical image segments;

[0037] The projection module is used to obtain the corresponding projection curve for each vertical segment image using the horizontal projection method, and to determine the character row area of ​​each row of characters based on the projection curve of the vertical segment image.

[0038] The region extraction module is used to extract a single character region from a character line region;

[0039] The recognition module is used to perform character recognition on a single character region on the steel pipe image using a preset recognition model to obtain recognition results.

[0040] On the other hand, this application also provides an electronic device, characterized in that it includes:

[0041] processor;

[0042] Memory used to store processor-executable instructions;

[0043] The processor is configured to perform the steps of the above-described method for detecting multi-line handwritten characters on steel pipes.

[0044] On the other hand, this application also provides a computer-readable storage medium storing computer program instructions thereon, characterized in that, when the program instructions are executed by a processor, they implement the steps of the steel pipe multi-line handwritten character detection method described in any of the above claims.

[0045] This application addresses the problem of multi-line handwritten character detection on steel pipes by proposing a segmentation-projection method for multi-line tilted handwritten character segmentation. Through single-line segmentation, character segmentation, and character recognition of the multi-line character region, this application achieves effective recognition and detection of multi-line handwritten characters on steel pipes. It is applicable to the recognition of handwritten characters on steel pipes in actual production processes and can efficiently and accurately identify handwritten characters on steel pipes. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the steel pipe manufacturing process in the background art.

[0047] Figure 2 This is one of the flowcharts for the method of detecting multi-line handwritten characters on steel pipes in the embodiments of this application.

[0048] Figure 3 This is the second flowchart of the method for detecting multi-line handwritten characters on steel pipes in the embodiments of this application.

[0049] Figure 4 This is the third flowchart of the method for detecting multi-line handwritten characters on steel pipes in the embodiments of this application.

[0050] Figure 5 This is the fourth flowchart of the method for detecting multi-line handwritten characters on steel pipes in the embodiments of this application.

[0051] Figure 6 This is the fifth flowchart of the method for detecting multi-line handwritten characters on steel pipes in the embodiments of this application.

[0052] Figure 7 This is the sixth flowchart of the method for detecting multi-line handwritten characters on steel pipes in the embodiments of this application.

[0053] Figure 8 This is a schematic diagram comparing the processing results of three segmentation algorithms in the embodiments of this application.

[0054] Figure 9 This is a schematic diagram of the segmentation of multiple lines of handwritten characters in different states using the horizontal projection method in an embodiment of this application.

[0055] Figure 10 This is an example image of grayscale image segmentation and its horizontal projection histogram in the embodiments of this application.

[0056] Figure 11 This is a schematic diagram of the fitting of the peak points of each row of characters in multiple vertical segment images in the embodiments of this application.

[0057] Figure 12 This is a schematic diagram of each channel of the RGB color space and LAB color space of the image of the character line area in the embodiments of this application.

[0058] Figure 13 These are multiple images used in the process of extracting edge features of characters in the L-channel image in this embodiment of the application.

[0059] Figure 14 This is a grayscale image of character segmentation in the embodiments of this application.

[0060] Figure 15 This is a schematic diagram of a segmented single character region in an embodiment of this application.

[0061] Figure 16 This is a schematic diagram of the network structure of the improved LeNet-5 model in the embodiments of this application.

[0062] Figure 17 This is a schematic diagram comparing the training and testing results of the improved LeNet-5 model and the original LeNet-5 model in the embodiments of this application.

[0063] Figure 18 This is a schematic block diagram of the steel pipe multi-line handwritten character detection device in the embodiments of this application. Detailed Implementation

[0064] The following are specific embodiments of this application, described in conjunction with the accompanying drawings, to further illustrate the technical solutions of this application. However, this application is not limited to these embodiments. In the following description, specific details such as particular configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Furthermore, for clarity and brevity, descriptions of known functions and structures have been omitted.

[0065] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0066] refer to Figure 2 This application provides a method for detecting multi-line handwritten characters on steel pipes, which can be applied to the detection of multi-line handwritten characters in complex backgrounds during steel pipe production processes. The method includes steps S101 to S106, and will be described in detail below with reference to the accompanying drawings.

[0067] Step S101: Extract the multi-line handwritten character region on the steel pipe image to obtain a character region image containing multiple lines of handwritten characters.

[0068] Step S102: Convert the character region image into a grayscale image.

[0069] Step S103: The grayscale image is divided into equal segments along the vertical direction to obtain multiple vertical segment images.

[0070] Step S104: For each vertical segment image, a horizontal projection method is used to obtain the corresponding projection curve, and the character row area of ​​each row of characters is determined according to the projection curve of the vertical segment image.

[0071] Step S105: Extract a single character region from the character line region.

[0072] Step S106: Use a preset recognition model to perform character recognition on a single character region on the steel pipe image to obtain the recognition result.

[0073] In this embodiment, the above-described method for detecting multi-line handwritten characters on a steel pipe is applied to an electronic device, which includes at least a camera, a processor, and a memory. The camera is used to capture images of the steel pipe containing multi-line handwritten characters, the memory is used to store related data, and the processor is used to process the images.

[0074] In step S101, a camera is first used to capture an image of the steel pipe, and a multi-line handwritten character region on the steel pipe image is extracted to obtain a character region image containing multiple lines of handwritten characters.

[0075] In a specific implementation, the extraction of the multi-line handwritten character region on the steel pipe image specifically involves using the YOLOv5 object detection algorithm to extract the multi-line handwritten character region on the steel pipe image. This application uses YOLOv5 to locate the multi-line character region in the acquired image, accurately obtaining the multi-line handwritten character region even in complex backgrounds. YOLOv5 is a classic object detection algorithm in deep learning, capable of efficiently extracting the multi-line handwritten character region required in this paper.

[0076] In step S102, the character region image is converted into a grayscale image. After obtaining the multi-line handwritten character region image of the steel pipe from YOLOv5, image preprocessing is performed. First, the color image is converted into a grayscale image. This step can use HSV thresholding, Canny edge segmentation, and Otsu segmentation algorithms. The resulting image is shown below. Figure 8 As shown in the figure, the segmentation results indicate that HSV threshold segmentation is the best method for image preprocessing.

[0077] In steps S103 and S104, this application provides a segmentation-improved projection method. First, the image is uniformly segmented vertically. Then, a horizontal projection method is applied to each small image to obtain the peak points of each row of characters. If characters in the horizontal direction still overlap in a segmented small image, the uniform segmentation factor is increased until each image obtains at least the required number of peak points. Finally, the peak points of each row of each small image are matched one-to-one to form peak fitting lines. A schematic diagram of the fitting curve is shown below. Figure 11 As shown.

[0078] refer to Figure 3 In one embodiment of this application, step S104, determining the character row region of each row of characters based on the projection curve of the vertical segment image, includes:

[0079] Step S1041: Determine the peak points of each line of characters in the projection curve.

[0080] Step S1042: Match the peak points of each row of characters in the multiple vertical segment images to obtain the character row region of each row of characters.

[0081] In steps S1041 and S1042, peak points in each row of each small image are matched one-to-one to form peak fitting lines, as shown in the schematic diagram of the fitting curve. Figure 11 As shown.

[0082] It should be understood that the projection-based character segmentation method segments the preprocessed character region according to the distribution pattern of peaks and troughs. It uses the peak points on the histogram curve that satisfy given height and width thresholds as segmentation points, and then segments a single line of characters based on these points, achieving the desired effect. Figure 9 As shown in (a). However, when multiple lines of handwritten characters are tilted at different angles, the horizontal projection method cannot be directly used to segment the character lines. Even if some multi-line characters can obtain the required number of segmentation points, as shown in (a). Figure 9 (b) It can be seen that the horizontal projection method has a very poor segmentation effect and cannot be applied to the multi-line handwritten character segmentation in this paper.

[0083] Furthermore, when an image is uniformly divided into multiple images, the character rows in each individual image will again become approximately horizontal, and there will be no more overlapping characters. Moreover, the horizontal projection method can once again achieve the required number of segmentation points. Example images of the segmentation and their horizontal projection histograms are shown below. Figure 10 As shown.

[0084] Further reference Figure 4In one embodiment of this application, step S1042, which involves matching the peak points of each row of characters in the plurality of vertical segment images to obtain the character row region of each row of characters, includes:

[0085] Step S1043: Fit the peak points of each row of characters in multiple vertical segment images to obtain several peak fitting lines.

[0086] Step S1044: Take the middle dividing line at the maximum width position between adjacent peak fitting lines as the dividing line of adjacent character line regions, with each peak fitting line corresponding to one character line region; generate the upper edge line of the uppermost character line region above the uppermost peak fitting line, and the distance between the upper edge line and the adjacent dividing line below is equal to the width at the maximum width position between the two uppermost peak fitting lines; generate the lower edge line of the lowermost character line region below the lowermost peak fitting line, and the distance between the lower edge line and the adjacent dividing line above is equal to the width at the maximum width position between the two lowermost peak fitting lines.

[0087] Step S1045: Determine the character line area of ​​each line of characters based on the dividing line of adjacent character line areas, the upper edge line of the uppermost character line area, and the lower edge line of the lowermost character line area.

[0088] Specifically, the peak fitting line is as follows: Figure 11 As shown, the peak fitting lines extend roughly horizontally, and their widths may vary depending on their positions. In step S1044, the midpoint of the maximum width between adjacent peak fitting lines is taken as the dividing line between adjacent character line regions. Then, the upper edge line of the uppermost character line region is generated above the uppermost peak fitting line, and the lower edge line of the lowermost character line region is generated below the lowermost peak fitting line. Finally, the character line region of each line of characters is determined based on the dividing lines between adjacent character line regions, the upper edge line of the uppermost character line region, and the lower edge line of the lowermost character line region. For a detailed execution flowchart, please refer to [link to flowchart]. Figure 7 Additionally, the angle of the dividing line can be determined based on the angle of the peak fitting line.

[0089] refer to Figure 5 In one embodiment of this application, step S105, extracting a single character region from the character line region, includes:

[0090] Step S1051: Convert the image of each character line region to the LAB color space and extract the L channel image.

[0091] Step S1052: Perform median filtering on the L channel image.

[0092] Step S1053: Extract the edge features of the characters in the L channel image to obtain an edge feature image.

[0093] Step S1054: Perform morphological dilation on the edge feature image, then fill it to obtain a filled character image, and remove the small white areas caused by noise.

[0094] Step S1055: Perform connected component extraction on the filled character image to obtain segmented individual character regions.

[0095] Specifically, in the LAB color space, the L channel represents luminance, and the A and B channels represent chroma; luminance and chroma channels are separate. In contrast, in the RGB color space, the R, G, and B channels are affected by both luminance and chroma simultaneously. (See reference...) Figure 12 Observe the RGB (corresponding) of the image Figure 12 ,ac) and LAB color space (corresponding to Figure 12 (df). When the image is in LAB color space, the corresponding Figure 12 In the digital form (df), the edge features of the characters are almost entirely located in the L-channel image. Therefore, this paper converts the obtained single-line handwritten character image into the LAB color space and extracts the L-channel image for subsequent processing. The obtained L-channel image is first subjected to median filtering to remove noise caused by factors such as the camera during the shooting process, and then the Canny edge operator is used to extract edge features.

[0096] Further reference Figure 6 Step S1053, extracting the edge features of characters in the L-channel image to obtain an edge feature image, includes:

[0097] Step S1056: Use the Canny edge operator to extract edge features from the L channel image to form the first image.

[0098] Step S1057: Set all non-black pixels in the L channel image to white to form a binary image, denoted as the second image.

[0099] Step S1058: Perform morphological erosion operation on the second image to obtain the third image.

[0100] Step S1059: Take the intersection of the first image and the third image to obtain the fourth image, which is used as the edge feature image.

[0101] Specifically, directly extracting edge features would include the boundary between the black border shown in the image and the actual image, while directly applying perspective transformation would affect the shape of the actual characters. Therefore, to remove erroneous edge detection at the boundary of the black border in the image without affecting the shape of the original characters, a reference is made... Figure 13Image 1 is the image obtained by the Canny edge operator extraction. Image 2 is the binary image formed by setting all non-black pixels in the L channel image to white. Image 3 is obtained by performing morphological erosion on image 2. Image 4 is obtained by finding the intersection of image 3 and image 1. The above images are in... Figure 13 As shown, image 1 corresponds to the first image, image 2 corresponds to the second image, image 3 corresponds to the third image, and image 4 corresponds to the fourth image. Finally, the above steps remove the edge lines detected by the Canny edge operator at the black border boundaries of the images without changing the character shape.

[0102] It's important to understand that the character line region cropping in step S1042 is not done using a standard rectangular cropping method, but rather an irregular quadrilateral cropping method. This results in a black border. At the boundary between the real image and the black border, the pixel value gradient is significant, so the Canny edge operator identifies it as an edge feature, forming a black border. Steps S1056 to S1059 are used to eliminate this black border in order to obtain the correct edge feature image.

[0103] Morphological dilation is performed on the processed edge feature image, followed by padding to obtain the padded character image. Small white areas caused by noise are removed using an area threshold, and finally, a grayscale image of the character segmentation is obtained, as shown below. Figure 14 As shown. Finally, connected component extraction is performed on the filled character image to obtain segmented character regions, including images of single characters and connected characters, as shown. Figure 15 As shown.

[0104] refer to Figure 16 In one embodiment of this application, the recognition model is an improved LeNet-5 model; the improved LeNet-5 model adds a self-attention mechanism to the LeNet-5 model, changes the last activation function to use the h-swish activation layer of MobileNetV3, and adds a Dropout layer.

[0105] Specifically, the LeNet-5 model is a classic handwritten character recognition model, consisting of two convolutional layers, two pooling layers, two fully connected layers, and one output layer. To improve the performance of the LeNet-5 model, this application proposes an improved model. First, it adopts the currently popular self-attention mechanism to enhance the model's perceptual ability. The final activation function is changed to the h-swish activation layer from MobileNetV3. The swish activation function has better performance than ReLU(), and the h-swish activation layer uses an approximation function to approximate the swish function, preserving its performance while reducing computational overhead. A Dropout layer is added; by discarding some neurons, the Dropout layer reduces the probability of overfitting and improves the model's generalization ability and robustness. Finally, considering that three fully connected layers increase computational overhead with minimal performance improvement, one fully connected layer is removed. The improved model network structure is shown below. Figure 16 As shown.

[0106] To evaluate the performance of the proposed improved LeNet-5 convolutional neural network, experiments were conducted using a dataset of character images obtained through the character segmentation method presented in this paper. The improved LeNet-5 convolutional neural network was written in Python and trained using the PyTorch framework; it ran on a Windows 10 environment; the GPU was an NVIDIA GeForce GTX 1650, and the RAM was 16GB; the network did not have a fixed learning rate. For the first 550 iterations, the learning rate and momentum were set to 0.001, and for the last 100 iterations, the learning rate was set to 0.0001, with no momentum parameter set; the cross-entropy loss function was used; five sets of comparative experiments were conducted under the same experimental environment. The first set used the original LeNet-5 model; the second set used the LeNet-5-Increase-Channel model, which increased the number of convolutional kernels in the convolutional layers compared to the original LeNet-5 model, changing from 6 and 16 to 16 and 32; the third set... The first group of experiments used the LeNet-5-Reduce-KennelSize model, which changed the kernel size of the original LeNet-5 model to a 3×3 kernel size. The fourth group of experiments used the LeNet-5-IR model, which combined the improvements of the second and third groups, increasing the number of kernels and reducing their size. The fifth group of experiments used the Gaijin_LeNet-5 model, which added a self-attention mechanism and a Dropout layer to the LeNet-5-IR model in the fourth group, and replaced the last ReLU activation function with the h-swish activation function.

[0107] Furthermore, the dataset consists of seven images of characters on steel pipes collected on-site by a mobile phone at a steel pipe manufacturing plant. The multi-line segmentation-projection method presented in this paper was used to process these seven images, resulting in 28 single-line character images. These single-line characters were then segmented to obtain 223 individual character images. An example of the extraction results is shown above. Figure 13 As shown, the extracted character images include single characters, connected characters, and missegmented characters. Images of different connected characters are treated as alternative single character images, and missegmented images are also treated as alternative single character images. Finally, all character images are classified into their corresponding character folders as the dataset for the character recognition model in this paper, totaling 31 classes. Simultaneously, to address the imbalance in the number of samples across character classes, image augmentation is performed on the data samples to improve the model's generalization ability. Noise is added and brightness is adjusted to obtain new images. By adding noise and adjusting brightness to each single character image, seven times more new images are generated. Finally, combined with the original images, a total of 1784 image samples are obtained. To test the effectiveness of the improved LeNet-5 network, the 1784 images are randomly divided into training and test sets in a 7:3 ratio and input into the network for training.

[0108] Furthermore, considering that character segmentation methods may result in three outcomes: too many segments, too few segments, or incorrect segments, this paper defines two evaluation metrics: segmentation precision and segmentation recall, to assess the results of the proposed character segmentation method. Segmentation precision is the ratio of the number of extracted character images that are also the characters to be extracted to the total number of extracted character images. Recall is the ratio of the number of extracted character images that are also the characters to be extracted to the total number of character images to be extracted. Higher precision and recall are desirable, ranging from [0,1]. To evaluate the performance of the proposed recognition model, experiments were conducted using a dataset created from single character images obtained through the proposed character segmentation method. Recognition accuracy was then chosen as the metric for evaluating the model's performance; it is calculated as the number of correctly recognized images divided by the total number of images.

[0109] The dataset was input into five different models for training, and the resulting model accuracy results are shown in the figure below. Figure 17 As shown in the diagram, comparing the second and third sets of experiments with the first set, we can see that the accuracy of the second and third sets is improved compared to the first set. Then, comparing the second and third sets with the fourth set, we can see that the fourth set, which combines the improvements from the second and third sets, has a higher accuracy than the first, second, and third sets, verifying the effectiveness of combining the increases in the number of convolutional kernels and the reduction in kernel size. Finally, comparing the fifth set with the other four sets, we can see that the fifth set, i.e., the Gaijin_LeNet-5 model in this paper, has a higher accuracy than the other four sets, verifying the effectiveness of the improved method presented in this paper.

[0110] This application trains and tests the enhanced dataset by inputting it into both the improved LeNet-5 model and the original LeNet-5 model, and finally obtains the recognition accuracy results on the test set as follows: Figure 17 As shown in the figure, the improved LeNet-5 model in this paper achieves a significantly higher recognition accuracy than the original LeNet-5 model, and its curve is also more stable. This application addresses the problem of multi-line handwritten character detection on steel pipes in complex backgrounds by proposing a segmentation-projection method for multi-line slanted handwritten character segmentation. The improved LeNet-5 recognition model is used to recognize individual character regions. Validated on the generated character image dataset, the average recognition accuracy reaches 0.97048, while the average accuracy of LeNet-5 is only 0.95947, a difference of 0.01101, thus verifying the effectiveness of the improved model. Through single-line segmentation, character segmentation, and character recognition of multi-line character regions, effective recognition and detection of multi-line handwritten characters on steel pipes in complex backgrounds is finally achieved.

[0111] This application addresses the problem of multi-line handwritten character detection on steel pipes by proposing a segmentation-projection method for multi-line tilted handwritten character segmentation. Through single-line segmentation, character segmentation, and character recognition of the multi-line character region, this application achieves effective recognition and detection of multi-line handwritten characters on steel pipes. It is applicable to the recognition of handwritten characters on steel pipes in actual production processes and can efficiently and accurately identify handwritten characters on steel pipes.

[0112] Furthermore, existing recognition models require a large number of samples for training to recognize handwritten digits, which is unsuitable for situations with a limited number of steel pipe images. The detection method provided in this application, however, is applicable to situations with a limited number of steel pipe image samples.

[0113] refer to Figure 18 This application also provides a steel pipe multi-line handwritten character detection device, including: an image extraction module 1801, a conversion module 1802, a segmentation module 1803, a projection module 1804, a region extraction module 1805, and a recognition module 1806.

[0114] Image extraction module 1801 is used to extract the multi-line handwritten character region on the steel pipe image to obtain a character region image containing multiple lines of handwritten characters.

[0115] The conversion module 1802 is used to convert the character region image into a grayscale image.

[0116] The segmentation module 1803 is used to segment the grayscale image vertically at equal intervals to obtain multiple vertical segment images.

[0117] The projection module 1804 is used to obtain the corresponding projection curve for each vertical segment image by using the horizontal projection method, and to determine the character row area of ​​each line of characters based on the projection curve of the vertical segment image.

[0118] The region extraction module 1805 is used to extract a single character region from a character line region.

[0119] The recognition module 1806 is used to perform character recognition on a single character region on the steel pipe image using a preset recognition model to obtain recognition results.

[0120] In one embodiment of this application, the projection module 1804 is specifically used for

[0121] Determine the peak points of each line of characters in the projection curve;

[0122] The peak points of each row of characters in the multiple vertical segment images are matched to obtain the character row region of each row of characters.

[0123] In one embodiment of this application, the projection module 1804 is specifically used for:

[0124] The peak points of each row of characters in multiple vertical image segments are fitted to obtain several peak fitting lines;

[0125] The middle dividing line at the maximum width position between adjacent peak fitting lines is taken as the dividing line of adjacent character line regions, and each peak fitting line corresponds to a character line region; the upper edge line of the uppermost character line region is generated above the uppermost peak fitting line, and the distance between the upper edge line and the adjacent dividing line below is equal to the width at the maximum width position between the two uppermost peak fitting lines; the lower edge line of the lowermost character line region is generated below the lowermost peak fitting line, and the distance between the lower edge line and the adjacent dividing line above is equal to the width at the maximum width position between the two lowermost peak fitting lines.

[0126] The character line area of ​​each line of characters is determined by the dividing line between adjacent character line areas, the upper edge of the uppermost character line area, and the lower edge of the lowermost character line area.

[0127] In one embodiment of this application, the region extraction module 1805 is specifically used for:

[0128] Convert the images of each character line region to the LAB color space and extract the L channel image;

[0129] The L-channel image is subjected to median filtering.

[0130] Extract the edge features of characters in the L-channel image to obtain an edge feature image;

[0131] Morphological dilation is performed on the edge feature image, and then filling is performed to obtain the filled character image, and small white areas caused by noise are removed;

[0132] Connected component extraction is performed on the filled character image to obtain segmented individual character regions.

[0133] In one embodiment of this application, the region extraction module 1805 is specifically used for:

[0134] The edge features in the L-channel image are extracted using the Canny edge operator to form the first image;

[0135] All non-black pixels in the L-channel image are set to white to form a binary image, which is denoted as the second image.

[0136] Perform morphological erosion on the second image to obtain the third image;

[0137] The intersection of the first and third images is used to obtain the fourth image, which is then used as the edge feature image.

[0138] In one embodiment of this application, the recognition model is an improved LeNet-5 model; the improved LeNet-5 model adds a self-attention mechanism to the LeNet-5 model, changes the last activation function to use the h-swish activation layer of MobileNetV3, and adds a Dropout layer.

[0139] In one embodiment of this application, the extraction of the multi-line handwritten character region on the steel pipe image specifically involves using the YOLOv5 object detection algorithm to extract the multi-line handwritten character region on the steel pipe image.

[0140] This application also provides an electronic device, including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the steps of the multi-line handwritten character detection method for steel pipes provided in the preceding section. Here, the electronic device can be a mobile terminal, a computer, etc. Related content can be found in the preceding section regarding the multi-line handwritten character detection method for steel pipes, and will not be repeated here.

[0141] This application also provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the multi-line handwritten character detection method for steel pipes provided in the preceding section. Related content can be found in the preceding section regarding the multi-line handwritten character detection method for steel pipes, and will not be repeated here.

[0142] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0144] The specific embodiments described herein are merely illustrative examples of the spirit of this application. Those skilled in the art to which this application pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of this application or exceeding the scope defined by the appended claims.

Claims

1. A method for detecting multi-line handwritten characters on steel pipes, characterized in that, include: Extract the multi-line handwritten character region from the steel pipe image to obtain a character region image containing multiple lines of handwritten characters; Convert the character region image into a grayscale image; The grayscale image is divided into multiple vertical segments at equal intervals along the vertical direction. For each vertical segment image, a horizontal projection method is used to obtain the corresponding projection curve, and the character row area of ​​each row of characters is determined according to the projection curve of the vertical segment image; Extract a single character region from a character line region; A preset recognition model is used to perform character recognition on a single character region on the steel pipe image to obtain the recognition result; in, The step of determining the character row region of each row of characters based on the projection curve of the vertical segment image includes: Determine the peak points of each line of characters in the projection curve; The peak points of each row of characters in the multiple vertical segment images are matched to obtain the character row region of each row of characters; The step of mapping the peak points of each row of characters in the plurality of vertical segment images to obtain the character row region of each row of characters includes: The peak points of each row of characters in multiple vertical image segments are fitted to obtain several peak fitting lines; The middle dividing line at the maximum width position between adjacent peak fitting lines is taken as the dividing line of adjacent character line regions, and each peak fitting line corresponds to a character line region; the upper edge line of the uppermost character line region is generated above the uppermost peak fitting line, and the distance between the upper edge line and the adjacent dividing line below is equal to the width at the maximum width position between the two uppermost peak fitting lines; the lower edge line of the lowermost character line region is generated below the lowermost peak fitting line, and the distance between the lower edge line and the adjacent dividing line above is equal to the width at the maximum width position between the two lowermost peak fitting lines. The character line area of ​​each line of characters is determined by the dividing line between adjacent character line areas, the upper edge of the uppermost character line area, and the lower edge of the lowermost character line area.

2. The method for detecting multi-line handwritten characters on steel pipes according to claim 1, characterized in that, Extracting a single character region from a character line region includes: Convert the images of each character line region to the LAB color space and extract the L channel image; The L-channel image is subjected to median filtering. Extract the edge features of characters in the L-channel image to obtain an edge feature image; Morphological dilation is performed on the edge feature image, and then filling is performed to obtain the filled character image, and small white areas caused by noise are removed; Connected component extraction is performed on the filled character image to obtain segmented individual character regions.

3. The method for detecting multi-line handwritten characters on steel pipes according to claim 2, characterized in that, The step of extracting edge features of characters in the L-channel image to obtain an edge feature image includes: The edge features in the L-channel image are extracted using the Canny edge operator to form the first image; All non-black pixels in the L-channel image are set to white to form a binary image, which is denoted as the second image. Perform morphological erosion on the second image to obtain the third image; The intersection of the first and third images is used to obtain the fourth image, which is then used as the edge feature image.

4. The method for detecting multi-line handwritten characters on steel pipes according to claim 1, characterized in that, The recognition model is an improved LeNet-5 model; the improved LeNet-5 model adds a self-attention mechanism to the LeNet-5 model, changes the last activation function to use the h-swish activation layer of MobileNetV3, and adds a Dropout layer.

5. The method for detecting multi-line handwritten characters on steel pipes according to claim 1, characterized in that, The extraction of the multi-line handwritten character region on the steel pipe image specifically involves using the YOLOv5 object detection algorithm to extract the multi-line handwritten character region on the steel pipe image.

6. A steel pipe multi-line handwritten character detection device, characterized in that, include: The image extraction module is used to extract the multi-line handwritten character region on the steel pipe image to obtain a character region image containing multiple lines of handwritten characters; The conversion module is used to convert the character region image into a grayscale image; The segmentation module is used to segment the grayscale image vertically at equal intervals to obtain multiple vertical image segments; The projection module is used to obtain the corresponding projection curve for each vertical segment image using the horizontal projection method, and to determine the character row area of ​​each row of characters based on the projection curve of the vertical segment image. The region extraction module is used to extract a single character region from a character line region; The recognition module is used to perform character recognition on a single character region on the steel pipe image using a preset recognition model to obtain recognition results; in, The step of determining the character row region of each row of characters based on the projection curve of the vertical segment image includes: Determine the peak points of each line of characters in the projection curve; The peak points of each row of characters in the multiple vertical segment images are matched to obtain the character row region of each row of characters; The step of mapping the peak points of each row of characters in the plurality of vertical segment images to obtain the character row region of each row of characters includes: The peak points of each row of characters in multiple vertical image segments are fitted to obtain several peak fitting lines; The middle dividing line at the maximum width position between adjacent peak fitting lines is taken as the dividing line of adjacent character line regions, and each peak fitting line corresponds to a character line region; the upper edge line of the uppermost character line region is generated above the uppermost peak fitting line, and the distance between the upper edge line and the adjacent dividing line below is equal to the width at the maximum width position between the two uppermost peak fitting lines; the lower edge line of the lowermost character line region is generated below the lowermost peak fitting line, and the distance between the lower edge line and the adjacent dividing line above is equal to the width at the maximum width position between the two lowermost peak fitting lines. The character line area of ​​each line of characters is determined by the dividing line between adjacent character line areas, the upper edge of the uppermost character line area, and the lower edge of the lowermost character line area.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the steps of the steel pipe multi-line handwritten character detection method according to any one of claims 1-5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the steps of the steel pipe multi-line handwritten character detection method according to any one of claims 1-5.

Images