Underground nixie tube character recognition method based on improved YOLOv7 and improved PaddleOcr
By improving the YOLOv7 and PaddleOcr models, combining the multi-head self-attention mechanism and CTC loss, the problem of misidentification and slow speed of character recognition in downhole digital tubes in a poor field of view environment is solved, and higher recognition accuracy and robustness are achieved. Real-time detection speed.
Patent Information
- Application Number
- CN202410961517.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
The existing underground digital tube character recognition methods are prone to misidentification or non-identification in environments such as mines and other environments with poor field of viewing conditions, and the recognition speed is slow.
Using the method of improving YOLOv7 and improving PaddleOcr, by embedding the MultiScale ConvBlock module in the YOLOv7 model and introducing a multi-head self-attention mechanism and a weighted combination of CTC loss and character-level cross-entropy loss in the PaddleOcr model, combined with the cross-fusion of Transformer and CNN, Conduct detection and recognition of underground digital tube characters.
It improves the recognition accuracy and robustness of downhole digital tube characters, reduces missed detection errors in small character lines and decimal points, and speeds up real-time detection speed.
Smart Images

Figure CN118942097A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to an underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr. Background Art
[0002] Downhole digital tubes are pipes used for underground gas and liquid transmission, and for covering signal transmission lines. LED screens are installed on the pipes to display information such as the airflow velocity in the pipes. Sometimes characters of different sizes are displayed, with small characters in the lower left corner of the screen and large characters on the right side of the screen, and also including decimal points.
[0003] Since these pipes are set up in the mine, they are easily affected by dust and other factors, resulting in poor vision, so the recognition of these characters may be misidentified or not recognized. These situations increase the difficulty of feature extraction, so it is necessary to design a more robust feature extraction method to adapt to the recognition of different characters. In addition, the underground environment may cause the characters to be blurred or broken. This is because the display principle of the characters to be recognized is different from that of traditional LED digital tubes, which increases the difficulty of digital recognition. And because it is a real-time detection speed, the detection speed of the algorithm must be fast enough. Summary of the invention
[0004] The technical problem to be solved by the present invention is that the existing character recognition method is prone to misrecognition or non-recognition in environments with poor visual conditions such as mines, and the recognition speed is slow.
[0005] To this end, the present invention provides an underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr.
[0006] The technical solution adopted by the present invention to solve the technical problem is:
[0007] A downhole digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr comprises the following steps:
[0008] S1, collects pictures with characters to form the data set D1;
[0009] S2, train the detection network model using the improved YOLOv7 model using the data set D1, and obtain the result graph after detection;
[0010] S3, graying the detected result image obtained in S2 to form a new data set D2;
[0011] S4, build the recognition network model through the improved PaddleOcr model, and use the data set D2 to train the recognition network model;
[0012] S5, collect pictures again, repeat steps S1-S3 to form a test set, and use the detection network model and the recognition network model to complete the detection and recognition tasks of the underground digital tube characters.
[0013] Furthermore, in step S2, the detection network model is an improved YOLOv7 model, and a MultiScale ConvBlock module is embedded in the ELAN module in the improved YOLOv7 model. The ELAN module is composed of three CBS modules and three MultiScaleConvBlock modules. The ELAN module splits the channels and then splices and fuses them, so that the downhole digital tube character detection network model can learn the effects of diversified feature information and multi-scale feature information.
[0014] Furthermore, in step S2, the loss function in the detection network model is Among them, L total is the total loss function, N is the number of detection layers, L conf is the confidence function, L reg is the regression loss function, L cls is the classification loss function, w1, w2, w3 are the weights of the above three loss functions respectively.
[0015] Furthermore, the confidence function L conf The expression is:
[0016]
[0017] Among them, y i ∈[0,1] represents the intersection over union (IoU) of the predicted target bounding box and the true target bounding box, σ represents the obtained prediction confidence, b represents the predicted box, b gt represents the label box, n is the number of positive and negative samples, x i is the probability of the current category after the activation function; the regression loss function L reg The WHLF weighted harmonic loss function is used, and its expression is: Where n is the number of samples; w i is the weight of the i-th sample; truncated_error i is the error after truncation, truncated_error i =min(abs(y i -f(x i )),τ),abs(y i -f(x i )) is the predicted value f(x i ) and the true value y iThe absolute error between τ and τ is a preset truncation threshold. When the error exceeds this threshold, it is truncated to τ to reduce the impact of extreme values on the loss function.
[0018] Furthermore, in step S4, the improved PaddleOcr model is a lightweight text recognition network that integrates a Transformer-based SVTR network and a lightweight CNN network PP-LCNet, and the Transformer layer in the improved PaddleOcr model introduces a multi-head self-attention mechanism.
[0019] Furthermore, the improved PaddleOcr model uses a weighted combination of CTC (Connectionist Temporal Classification) loss and character-level cross entropy loss.
[0020] Furthermore, the CTC loss function is defined as negative log-likelihood loss, i.e., CTCLoss = -logp(zx), which is optimized during training to maximize the conditional probability of the target sequence z (text character sequence) given the input sequence x (image feature sequence of the OCR task); the character-level cross entropy loss function is Where T is the total number of characters in the character sequence, is the predicted probability distribution of the tth character in the character sequence, is the true character sequence of the tth character in the character sequence; the loss function of the improved PaddleOcr model is Loss = w·CCELoss+(1-w)·CTCLoss, where w is the weight coefficient, w∈[0,1], and w can optimize the loss function parameters through genetic algorithm.
[0021] Furthermore, the process of updating the loss function parameters includes the following steps:
[0022] B1, use randomly initialized θ to complete the update of loss function parameters and obtain the initial population set;
[0023] B2, calculate each individual in the population set Adaptability;
[0024] B3, select the top 10% of individuals from the largest to the smallest fitness as excellent individuals to enter the next generation, and use the crossover operation to introduce new variation individuals;
[0025] B4, repeat B1-B3 until the number of selected excellent individuals is less than 6, and select the individual with the highest fitness from the final population as the parameter of the loss function.
[0026] Furthermore, in the improved PaddleOcr model, Transformer and CNN are cross-fused, the output of the convolutional layer of CNN is used as the input of Transformer, the weight coefficient w is used to control the proportion of Transformer, and the influence of Transformer on the original CNN layer is dynamically adjusted to achieve the complementarity of the two network structures.
[0027] Furthermore, the process of passing the CNN output to the Transformer includes the following steps:
[0028] C1, the output of the middle layer of CNN, generates a four-dimensional tensor after passing through the convolutional layer of CNN;
[0029] C2, data reshaping, reshaping the four-dimensional tensor output by CNN into a two-dimensional tensor;
[0030] C3, dimension adjustment, expands the two-dimensional tensor into a three-dimensional tensor by copying or filling, so that each pixel is represented as a vector of embedding_dim dimension;
[0031] C4, position coding, uses sine and cosine functions to generate position-related pixel encoding vectors;
[0032] C5 passes the 3D tensor in C3 and the encoded vector in C4 to the Transformer.
[0033] The beneficial effect of the present invention is that the present application improves the accuracy and robustness of detection and recognition on the basis of the original algorithm. The present application establishes an improved YOLOv7 target detection network, uses the MultiScaleConvBlock module to replace the original convolution layer in the backbone network for feature extraction, and replaces the bounding box regression function with the WHLF weighted harmonic loss function. After obtaining the detection results, the uneven luminescence of the characters is unified, and the detected text result image is binarized. Then an improved PaddleOcr text recognition network is established. A multi-head self-attention mechanism is introduced in the Transformer layer. A cross-fusion method of the Transformer layer and CNN is provided, and the output of the convolution layer of CNN is used as the input of the Transformer layer. The original text recognition loss function is replaced by a weighted combination of CTC loss and character-level cross entropy loss. Using the improved YOLOv7 and PaddleOcr models, the detection and recognition tasks of underground digital tube characters are completed respectively, and finally the real-time recognized characters are uploaded through fcp. The present invention can greatly improve the problems of missed detection of small character row targets, missed detection of decimal points and misidentification, and accelerates the speed of real-time detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0035] Figure 1 This is a flowchart of the improved PaddleOcr traffic target detection method based on improved YOLOv7 described in the present invention.
[0036] Figure 2 This is a structural diagram of the MultiScaleConvBlock module described in the present invention.
[0037] Figure 3 It is a structural diagram of the ELAN module described in the present invention.
[0038] Figure 4 This is a structural diagram of the cross-fusion method of Transformer and CNN described in the present invention.
[0039] Figure 5 This is a diagram of the recognition test results in the present invention.
[0040] Figure 6 This is a schematic diagram of the screen of the recipient of the real-time recognition result in the present invention. DETAILED DESCRIPTION
[0041] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.
[0042] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.
[0043] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0044] Reference Figure 1-6 , a downhole digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr, comprising the following steps,
[0045] S1, collects pictures with characters to form the data set D1
[0046] S2, the detection network model is trained using the improved YOLOv7 model using the dataset D1, and the result graph after detection is obtained
[0047] S2.1, build detection network model
[0048] The YOLOv7 model is improved by embedding the MultiScale ConvBlock module (custom multi-scale convolution block) in the ELAN module. Specifically, the ELAN module (as shown in the attached Figure 3 As shown in FIG. 1 , the network model for detecting downhole digital tube characters is composed of three CBS modules and three MultiScaleConvBlock modules. As for the ELAN module, it mainly splits the channels and then splices and fuses them, so that the downhole digital tube character detection network model can learn the effects of diversified feature information and multi-scale feature information.
[0049] The MultiScaleConvBlock module includes two convolutions with a kernel size of 5×5, two convolutions with a kernel size of 3×3, and two point-wise convolutions with a kernel size of 1×1. Figure 2In the sub-figure a, the input is first convolved with a convolution kernel size of 1×1 on the left branch to change the input channel Cin to Cout, and the input is sent to the right branch at the same time. Then, a convolution with a convolution kernel size of 5×5 is performed on the left sub-branch to change the channel from Cin to 0.5*Cout, and then a convolution with a convolution kernel size of 3×3 is performed to change the channel from 0.5*Cout to Cout. At the same time, a convolution with a convolution kernel size of 3×3 is performed on the right sub-branch to change the number of channels from Cin to 0.5*Cout. Then a convolution with a convolution kernel size of 5×5 is performed to change the channel from 0.5*Cout to Cout, and then the output of the left sub-branch is multiplied by the parameter ω=1 and added element by element to the output of the right sub-branch. Then the output is convolved with a convolution kernel size of 5×5, and the number of channels remains unchanged. The result is added element by element to the output of the left branch to obtain the module output.
[0050] In the inference stage, refer to Figure 2 As shown in the sub-figure b in the figure, the parameter ω is set to 0. That is, the input passes through the left branch convolution with a kernel size of 1×1 to convert the input channel Cin into Cout, and at the same time, the input is sent to the right branch for a convolution with a kernel size of 3×3, and then a convolution with a kernel size of 5×5. Then the left branch result is added element by element with the right branch result.
[0051] In addition, the loss function defined in the detection network model in the present invention is:
[0052]
[0053] Among them, L total is the total loss function, N is the number of detection layers, L conf is the confidence function, L reg is the regression loss function, L cls is the classification loss function, w1, w2, w3 are the weights of the above three loss functions respectively.
[0054] Confidence function L conf The expression is:
[0055]
[0056] In the above formula, y i ∈[0,1] represents the intersection over union (IoU) of the predicted target bounding box and the true target bounding box, σ represents the obtained prediction confidence, b represents the predicted box, b gt represents the label box, n is the number of positive and negative samples, x i is the probability of the current category after the activation function.
[0057] Regression loss function L regThe WHLF weighted harmonic loss function is used, and its expression is:
[0058]
[0059] n is the sample size; w i is the weight of the i-th sample; truncated_error i is the error after truncation, which is defined as follows:
[0060] truncated_error i =min(abs(y i -f(x i )),τ)
[0061] Here abs(y i -f(x i )) is the predicted value f(x i ) and the true value y i The absolute error between τ and τ is a preset truncation threshold. When the error exceeds this threshold, we truncate it to τ to reduce the impact of extreme values on the loss function.
[0062] The characteristic of this loss function is that when the error is small, due to the nature of the harmonic mean, the loss function is more sensitive to the change of the error, which helps the model to make more precise adjustments in areas with smaller errors. i It can be defined according to actual conditions to adjust the contribution of different samples in the loss function. The truncation mechanism can reduce the impact of extreme values on the loss function and improve the robustness of the model.
[0063] S2.2, input the graphic data in the data set D1 into the detection network model and train it, use the trained detection network model to detect the graphics in the data set D1, and obtain the result graph after detection.
[0064] S3, grayscale processing is performed on the detected result image obtained in S2 to form a new data set D2.
[0065] Because the light-emitting principle of the detected underground digital tube is different from that of the traditional LED digital tube, the light emission of different parts of the same character is not uniform. Blindly performing image processing or data addition cannot solve the problem well, and the workload is huge. Therefore, it is proposed to perform binarization processing on the detected images, unify the uneven light emission in various parts into a black color, and reduce the difficulty of recognition.
[0066] Definition of binarization: Set the grayscale value of the pixels on the image to 0 or 255, which means that the entire image appears to be only black and white.
[0067] The threshold of each pixel is determined based on the statistical information of the grayscale value of the local area around the pixel. Assuming that the size of the local area of the image is N×N, for each pixel point P(x,y), calculate the average grayscale value avg of its local area. Then the binary pixel value B(x,y) can be calculated by the following formula:
[0068]
[0069] For each pixel, first calculate the average gray value avg of its local area (such as an N×N window), and then compare the gray value of the pixel with avg. If the pixel gray value is greater than avg, the pixel value after binarization is 255 (white); otherwise, it is 0 (black).
[0070] During the actual recognition process, we tested many times and found that the recognition accuracy is higher in the case of black text and white background than in the case of white text and black background. Therefore, in the actual recognition process, avg is sometimes adjusted to convert all images into black text and white background.
[0071] S4, build the recognition network model through the improved PaddleOcr model, and use the data set D2 to train the recognition network model
[0072] S4.1 Construction of recognition network model
[0073] S4.1.1, Transformer layer improvements
[0074] The PaddleOcr model uses the SVTR_LCNet lightweight network model to recognize text, especially in the PP-OCRv3 version, SVTR_LCNet is used as the main text recognition network model. SVTR_LCNet is a lightweight text recognition network that integrates the Transformer-based SVTR network and the lightweight CNN network PP-LCNet. The present invention improves it from multiple aspects.
[0075] An attention mechanism, Multi-Head Self-Attention, is introduced in the Transformer layer to enhance the model's ability to focus on important information. It obtains the attention distribution of different subspaces of the input sequence by running multiple independent attention mechanisms in parallel, thereby more comprehensively capturing the potential multiple semantic associations in the sequence. In multi-head attention, the input sequence first passes through three different linear transformation layers to obtain the Query, Key, and Value respectively. Then, these transformed vectors are divided into several "heads", each with its own independent Query, Key, and Value matrices. For each head, a Scaled Dot-Product Attention operation is performed, namely:
[0076]
[0077] Finally, the outputs of all heads are concatenated together and then fused through a linear layer to obtain the final attention output vector.
[0078] In this way, multi-head attention can process the input sequence from different perspectives in parallel, improving the model's ability to understand and capture complex dependencies. In practice, multi-head attention can significantly improve the performance of the Transformer model in natural language processing and other sequence data processing tasks.
[0079] S4.1.2 Improvement of loss function
[0080] The improved PaddleOcr model in the present invention uses a weighted combination of CTC (Connectionist Temporal Classification) loss and character-level cross entropy loss to better guide model training.
[0081] The CTC (Connectionist Temporal Classification) loss function was originally used for speech modeling in speech recognition. Its purpose is to introduce CTC constraints in the neural network model so that the model can map the input sequence to the output sequence without knowing the correspondence between the input sequence and the output sequence.
[0082] The goal of the CTC loss function is to minimize the negative log-likelihood of the output sequence. The CTC algorithm consists of three main steps:
[0083] A1 merges repeated tokens in the target sequence into one token
[0084] A2 calculates all possible alignment paths for the target sequence
[0085] A3 merges duplicate markers in each alignment path into one marker and calculates the negative log-likelihood probability of the path
[0086] The CTC loss function is defined as the negative log-likelihood loss, i.e.:
[0087] CTCLoss=-logp(zx)
[0088] This loss function is optimized during training to maximize the conditional probability of the target sequence z (a sequence of text characters) given an input sequence x (a sequence of image features for the OCR task).
[0089] Character-level cross-entropy loss is usually used to process sequence data, such as character sequences in text data. In character-level classification tasks, each character is regarded as an independent category and a probability distribution is assigned to it.
[0090] Suppose we have a predicted probability distribution y for a character sequence pred and a real character sequence y true We can define the character-level cross entropy loss function as follows: The total number of characters in the character sequence is T, and for each character position t in the sequence, we have the predicted probability distribution This is a Softmax output vector, representing the probability of all possible characters), and the actual characters This is encoded as a one-hot vector where the position of the correct character is 1 and the rest are 0. The character-level cross entropy loss function can be expressed as:
[0091]
[0092] The loss function of the recognition task of the present invention is finally determined as:
[0093] Loss=w·CCELoss+(1-w)·CTCLoss
[0094] w is the weight coefficient, w∈[0,1], and w can optimize the loss function parameters through genetic algorithm.
[0095] The process of optimizing and updating the loss function parameters based on the genetic algorithm is as follows:
[0096] B1, use the randomly initialized vector θ to complete the update of the loss function parameters and obtain the initial population set:
[0097]
[0098] Where Y is the population size; τ = 1, 2,…, Y.
[0099] B2, for each individual in the population Calculate the fitness of the individual, and the fitness is calculated as follows:
[0100]
[0101] in, Indicates that based on individual The error between the recognition result of the calculated recognition network model and the true result;
[0102] B3, select the top 10% of individuals from the largest to the smallest fitness as excellent individuals to enter the next generation, and use the crossover operation to introduce new variation individuals. The crossover operation can be expressed as:
[0103]
[0104] Among them, ψ' is the set of excellent individuals entering the next generation; random(ψ') means randomly selecting individuals from ψ'. In the above formula, two random selections are made respectively. Because it is a random selection, the previous and the next are different; the crossover operation is used to expand the population to 40% of the population before selecting excellent individuals.
[0105] B4, repeat the above steps until the number of selected excellent individuals is less than 6, and select the individual with the highest fitness from the final population as the parameter of the loss function.
[0106] S4.1.3 Feature Fusion Innovation
[0107] Multi-scale feature fusion: Multi-scale feature fusion is performed at different levels of SVTR_LCNet to capture text information of different granularities. The Transformer module can perform sequence modeling on feature maps and convert text information in images into sequence data for subsequent text recognition. Through the self-attention mechanism, the Transformer can learn the dependencies between characters in the text sequence, thereby improving recognition accuracy.
[0108] Now we provide a cross-fusion method of Transformer and CNN, taking the output of the convolutional layer of CNN as the input of Transformer, using the weight coefficient w to control the proportion of Transformer, and dynamically adjusting the influence of Transformer on the original CNN layer to achieve the complementarity of the two network structures. Figure 5 shown.
[0109] In SVTR_LCNet, CNN is mainly used to extract visual features of images, including information such as the shape, outline, and strokes of text, providing a basis for subsequent text recognition. The CNN module in SVTR_LCNet may include multiple convolutional layers, which are stacked to gradually extract deep features in the image. As the network depth increases, CNN is able to capture higher-level features, including semantic information of text, contextual relationships, etc.
[0110] Transformer is a neural network model for processing sequence data. Its core is the self-attention mechanism, which can capture the dependencies between positions in the input sequence, thereby dealing with long-distance dependency problems. In SVTR_LCNet, the introduction of Transformer enables the model to better process the contextual information of text sequences and improve the accuracy of text recognition.
[0111] It should be noted that when the output of the convolutional layer of a CNN is used as the input of a Transformer, some conversion and adjustment are usually required to ensure that the data format and dimension match the input requirements of the Transformer. The following is a flow that illustrates how to pass the output of a CNN to a Transformer:
[0112] C1, the intermediate layer output of CNN
[0113] After passing through the convolutional layer of CNN, a four-dimensional tensor (Tensor) is generated, whose shape is [batch_size, height, width, channels]. Here, batch_size is the number of samples in the batch, height and width are the height and width of the feature map, and channels is the number of channels of the feature map.
[0114] C2, data reshaping
[0115] Since the Transformer layer in the present invention processes text sequence data, we need to reshape the four-dimensional tensor output by CNN into a two-dimensional tensor with a shape of [batch_size, sequence_length*channels], where sequence_length is the number of all pixels in the feature map (i.e., height*width).
[0116] C3, Dimension Adjustment
[0117] Before passing the data to the Transformer, further dimensionality adjustment is required. The input requirement of the Transformer is a 3D tensor with a shape of [batch_size, sequence_length, embedding_dim]. Here, embedding_dim is the dimension of the embedding vector, which is the size of the vector used inside the Transformer to represent each input token (or pixel). Therefore, we need to expand the 2D tensor obtained in the second step into a 3D tensor. This is achieved by copying or padding so that each pixel is represented as a vector of embedding_dim dimensions.
[0118] C4, positional encoding
[0119] Since Transformer is a self-attention based model, it does not directly contain the position information of the elements in the sequence. Therefore, in order to preserve the order information of the input data, that is, the relative position of the pixels in the image, we need to add position encoding to each input element. Use sine and cosine functions to generate position-related encoding vectors and add these vectors to the input data.
[0120] C5, passed to Transformer
[0121] Now, we have transformed the output of the CNN into a 3D tensor suitable for Transformer input and added positional encodings. This tensor can then be passed to the encoder part of the Transformer for further processing. The Transformer encoder will use the self-attention mechanism to capture the dependencies in the input data and generate an output representation that contains richer contextual information.
[0122] S5 collects pictures again and repeats steps S1-S3 to form a test set. The S detection network model and recognition network model are used to complete the detection and recognition tasks of the underground digital tube characters. Finally, the real-time recognized characters are uploaded through FCP.
[0123] Finally, the new data set after binarization is trained through the improved Paddleocr, and the trained recognition network model is used to recognize the pictures in the test set. The pictures in the test set are also tested by the improved YOLOv7 and then binarized. Then the recognition results are uploaded through fcp. Figure 5 As shown, Figure 5 Displays a screen with small characters ( Figure 5 a small picture in the figure) and the picture without small characters ( Figure 5In the uploaded text output, one line represents all the character recognition results of one test, including large and small characters.
[0124] In summary, the present application establishes an improved YOLOv7 target detection network, uses the MultiScaleConvBlock module to replace the original convolution layer in the backbone network for feature extraction, and replaces the bounding box regression function with the WHLF weighted harmonic loss function. After obtaining the detection results, the uneven luminescence of the characters is unified, and the detected text result image is binarized. Then an improved PaddleOcr text recognition network is established. A multi-head self-attention mechanism is introduced in the Transformer layer. A cross-fusion method of the Transformer layer and CNN is provided, and the output of the convolution layer of CNN is used as the input of the Transformer layer. The original text recognition loss function is replaced by a weighted combination of CTC loss and character-level cross entropy loss. Using the improved YOLOv7 and PaddleOcr models, the detection and recognition tasks of underground digital tube characters are completed respectively, and the real-time recognized characters are finally uploaded through fcp. The present invention can greatly improve the problems of missed detection of small character line targets, missed detection of decimal points and misrecognition, and speed up the speed of real-time detection. Based on the original algorithm, the accuracy and robustness of detection and recognition are improved.
[0125] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.
Claims
1. A method for underground digital tube character recognition based on improved YOLOv7 and improved PaddleOcr, It is characterized in that The following steps are included: S1, collect pictures with characters to form data set D 1 ; S2, using the improved YOLOv7 model using dataset D 1 Train the detection network model and obtain the result graph after detection; S3, grayscale the detected result image obtained in S2 to form a new data set D 2 ; S4, the recognition network model is constructed by improving the PaddleOcr model, and the dataset D 2 Train a recognition network model; S5, collect pictures again, repeat steps S1-S3 to form a test set, and use the detection network model and the recognition network model to complete the detection and recognition tasks of the underground digital tube characters.
2. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 1, It is characterized in that In step S2, the detection network model is an improved YOLOv7 model, and a MultiScale ConvBlock module is embedded in the ELAN module in the improved YOLOv7 model. The ELAN module is composed of three CBS modules and three MultiScaleConvBlock modules. The ELAN module splits the channels and then splices and fuses them, so that the downhole digital tube character detection network model can learn the effects of diversified feature information and multi-scale feature information.
3. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 1, It is characterized in that In step S2, the loss function in the detection network model is Among them, L total is the total loss function, N is the number of detection layers, L conf is the confidence function, L reg is the regression loss function, L cls is the classification loss function, w 1 , w 2 , w 3 are the weights of the above three loss functions respectively.
4. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 3, It is characterized in that Confidence function L conf The expression is: Among them, y i ∈[0,1] represents the intersection over union (IoU) of the predicted target bounding box and the true target bounding box, σ represents the obtained prediction confidence, b represents the predicted box, b gt represents the label box, n is the number of positive and negative samples, x i is the probability of the current category after the activation function; the regression loss function L reg The WHLF weighted harmonic loss function is used, and its expression is: Where n is the number of samples; w i is the weight of the i-th sample; truncated_error i is the error after truncation, truncated_error i =min(abs(y i -f(x i )),τ),abs(y i -f(x i )) is the predicted value f(x i ) and the true value y i The absolute error between τ and τ is a preset truncation threshold. When the error exceeds this threshold, it is truncated to τ to reduce the impact of extreme values on the loss function.
5. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 1, It is characterized in that In step S4, the improved PaddleOcr model is a lightweight text recognition network that integrates a Transformer-based SVTR network and a lightweight CNN network PP-LCNet. The Transformer layer in the improved PaddleOcr model introduces a multi-head self-attention mechanism.
6. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 5, It is characterized in that The improved PaddleOcr model uses a weighted combination of CTC (Connectionist Temporal Classification) loss and character-level cross entropy loss.
7. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 6, It is characterized in that The CTC loss function is defined as the negative log-likelihood loss, i.e., CTCLoss = -logp(z|x). This loss function is optimized during training to maximize the conditional probability of the target sequence z (text character sequence) given the input sequence x (image feature sequence of the OCR task); the character-level cross entropy loss function is Where T is the total number of characters in the character sequence, is the predicted probability distribution of the tth character in the character sequence, is the true character sequence of the tth character in the character sequence; the loss function of the improved PaddleOcr model is Loss = w·CCELoss+(1-w)·CTCLoss, where w is the weight coefficient, w∈[0,1], and w can optimize the loss function parameters through genetic algorithm.
8. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 7, It is characterized in that The process of updating the loss function parameters includes the following steps: B1, use randomly initialized θ to complete the update of loss function parameters and obtain the initial population set; B2, calculate each individual in the population set Adaptability; B3, select the top 10% of individuals from the largest to the smallest fitness as excellent individuals to enter the next generation, and use the crossover operation to introduce new variation individuals; B4, repeat B1-B3 until the number of selected excellent individuals is less than 6, and select the individual with the highest fitness from the final population as the parameter of the loss function.
9. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 8, It is characterized in that In the improved PaddleOcr model, Transformer and CNN are cross-fused, the output of the convolutional layer of CNN is used as the input of Transformer, the weight coefficient w is used to control the proportion of Transformer, and the influence of Transformer on the original CNN layer is dynamically adjusted to achieve the complementarity of the two network structures.
10. The underground digital tube character recognition method based on improved YOLOv7 and improved PaddleOcr according to claim 9, It is characterized in that The process of passing the CNN output to the Transformer includes the following steps: C1, the output of the middle layer of CNN, generates a four-dimensional tensor after passing through the convolutional layer of CNN; C2, data reshaping, reshaping the four-dimensional tensor output by CNN into a two-dimensional tensor; C3, dimension adjustment, expands the two-dimensional tensor into a three-dimensional tensor by copying or filling, so that each pixel is represented as a vector of embedding_dim dimension; C4, position coding, uses sine and cosine functions to generate position-related pixel encoding vectors; C5 passes the 3D tensor in C3 and the encoded vector in C4 to the Transformer.
Citation Information
Patent Citations
Time sequence attention mechanism scene image recognition method
CN113688822A
Digital identification method in nixie tube
CN115205870A
Converter station digital instrument identification method based on deep learning and OpenCV
CN115471845A
Weighted decision fusion identification method for real object ID tag
CN118154833A
Terminal block coding tube text detection and identification method based on deep learning
CN118155219A
