An industrial character image repairing method based on feature recognition mechanism

Abstract

This invention belongs to the field of text recognition and image restoration in industrial production processes, and discloses an industrial text image restoration method based on a feature recognition mechanism. The method proposed in this invention solves the problem of data waste caused by damaged images in slab number recognition systems during steel plate production. Through the image restoration method based on the feature recognition mechanism proposed in this invention, the slab number recognition system can obtain high-quality training and inference data, thereby significantly improving the accuracy of slab number recognition. Benefiting from the improved accuracy of slab number recognition, steel production enterprises can achieve accurate steel material tracking.

Description

Technical Field

[0001] This invention relates to the field of text recognition and image restoration in industrial production processes, and particularly to an industrial text image restoration method based on feature recognition mechanisms. Background Technology

[0002] In steel production, the slab number is a unique identifier for tracking the entire production process. Steel plate ID or slab number recognition is the foundation of intelligent steel tracking; large steel enterprises rely on slab number recognition systems for their information-based material tracking. In similar text recognition tasks, researchers have developed many mature methods, which can be categorized into traditional methods based on character features and recognition methods based on convolutional neural networks. The latter has better application results but relies on a large amount of high-quality data.

[0003] However, due to limitations in the printing performance of inkjet printing equipment and interference from the production environment, image data for character recognition tasks is easily damaged, leading to significant data waste and difficulty in obtaining sufficient high-quality data. For example, specular reflection on the steel plate surface can cause text to be obscured, resulting in poor plate number recognition when the text content is covered.

[0004] To date, there has been no research on text image repair in industrial production, particularly in the steel industry. The resulting image data defects in the current production environment significantly reduce the image quality input to slab number recognition systems, increasing the difficulty of recognition and leading to lower accuracy in slab number recognition, making it difficult to achieve precise tracking of steel. Summary of the Invention

[0005] Therefore, it is necessary to propose an industrial text image repair method based on feature recognition mechanism to address the above problems. This method can repair damaged text images and provide high-quality data for slab number recognition systems.

[0006] The technical solution of this invention is as follows: An industrial text image repair method based on feature recognition mechanism, comprising the following steps:

[0007] Step 1: Obtain industrial text image data from the slab number recognition system, and prepare defect training dataset and defect test dataset; add the content information and location information of the text region of the industrial text image data to the data label;

[0008] Step 2: Locate the defective region in the image to be repaired in the defective training dataset; obtain the defective region in the image to be repaired and generate a difference matrix by calculation;

[0009] Step 3: Perform feature identification using the difference matrix obtained in Step 2 to narrow down the defect area and fill in the feature value V. i Repeat steps 2 and 3 to obtain feature map group F;

[0010] Step 4: Perform a fusion operation on the feature map group F to obtain feature map F. m F is achieved through two fully connected convolutional layers. m Mapped to a sequence of feature vectors

[0011] Step 5: Process the feature vector sequence obtained in Step 4 using a bidirectional long short-term memory network. The sequences are reordered to obtain a new sequence. An attention mechanism is then used to decode the new sequence to obtain the weight α and feature value V for each element. h ;

[0012] Step 6: Apply weight α to the eigenvalues ​​V obtained in Step 5. h Perform secondary feature recognition, output the text content, and compare it with the text region content information and location information in the data label described in step 1; based on the comparison results, obtain the reconstructed image after two feature recognitions, or obtain the reconstructed image after 2n feature recognitions through n iterations of steps 2 to 6; the reconstructed image is the final repaired industrial text image.

[0013] Step 1 specifically involves:

[0014] Industrial text image data is acquired from the slab number recognition system. Defective text images that the OCR system cannot recognize are collected, and a binary pixel mask is input to overlay intact text images. The text region is a region different from the target repair area. The defective text images that the OCR system cannot recognize are those caused by excessive lighting, partial occlusion, etc., and are denoted as I. da ∈R, the intact text image is I gt ∈R, binary pixel mask is M∈{0,1}, and the superimposed image is I in ∈R, where I in =I gt ⊙M. Will I in with I da Random fusion is performed to obtain a defective training dataset and a defective test dataset.

[0015] Step 2 specifically involves:

[0016] By traversing the images to be repaired in the dataset through g-layer local convolutional layers, information about the missing regions is obtained, including the shape, size, location, and holes at the boundary of the images to be repaired for abnormal pixels. The feature value of the abnormal pixel position (x, y) in the i-th local convolutional layer is multiplied pixel by pixel by the m-th convolutional kernel to obtain the updated mask value, represented as:

[0017]

[0018] The difference matrix between the updated mask and the input mask is defined as the defective region to be inferred in the defective image; finally, after processing by a fully connected network layer and an activation function, the processed difference matrix is ​​used for a first feature recognition.

[0019] Step 3 specifically involves:

[0020] The first feature recognition is performed on the difference matrix region; the difference matrix is ​​then input into the knowledge consistency attention module to calculate the feature value V of the mask region updated in step 2. i ={v i |i=1,2,3,…,d}; Then, repeat steps 2 and 3, and use the feature values ​​v calculated by the knowledge consistency attention module. i The feature map f is obtained by filling the difference matrix and then pooling. pool Continue this process until no more missing areas are detected in step 2, then record the resulting feature map set as...

[0021] Step 4 specifically involves:

[0022] The feature map group F is obtained by fusing the feature map group F. m F is achieved through two fully connected convolutional layers. m The mapping is a sequence of feature vectors, denoted as . Each feature vector In the original feature map F m They are arranged horizontally from left to right.

[0023] Step 5 specifically involves:

[0024] Using a bidirectional long short-term memory network module Calculation yields the new sequence H = {h} h The internal correlation degree R = {r | h = 1, 2, 3, ..., j} and H is {r} h |h=1,2,3,...,j};

[0025] The calculation process is represented as: r h =BLSTM(r h-1 ,g h ,h h The alignment factor α is obtained by scoring each element in the new sequence H using an attention function and then normalizing the results. h G = {g} h =α h *h h |h=1,2,3,...,j} is the weighted sum of the new sequence H;

[0026] After obtaining the new sequence H and its internal correlation, the attention function is used to perform secondary decoding on the new sequence H to obtain the feature value V of each sequence element h. h .

[0027] Step 6 specifically involves:

[0028] The eigenvalues ​​V obtained in step 5 are weighted according to weight α. h The output is text content, which is compared with the text region content information and text information in the data labels described in step 1. If they are equal, the low-level high-resolution features and high-level semantic features of feature map group F are adaptively fused to obtain the reconstructed image, where the adaptive fusion uses the FPN algorithm; if they are not equal, steps 2 to 6 are iterated n times to obtain the reconstructed image; the iteration continues until the loss function L is reached. total Below the set threshold, where the loss function L required for iteration total It is expressed as follows:

[0029] L total =λ style L style +λ perceptual L perceptual +λ predicted L predicted (2)

[0030] Where λ is a learnable parameter, and L is the style loss. style Perceived loss L perceptual And predicted loss L predicted They are represented as follows:

[0031]

[0032]

[0033]

[0034] in, Let θ represent the true value in the label from step 1, and let θ represent the vector of all parameters in the feature recognition mechanism. Feature vector sequence, The i-th feature map after pooling is represented by H, W, and C, which represent the length, width, and number of channels of the input image, respectively.

[0035] The beneficial effects of this invention are:

[0036] The method proposed in this invention solves the problem of data waste caused by damaged images in the slab number identification system during steel plate production. Through the image repair method based on feature recognition mechanism proposed in this invention, the slab number identification system can obtain high-quality training and inference data, thereby significantly improving the accuracy of slab number identification. Benefiting from the improved accuracy of slab number identification, steel production enterprises can achieve accurate tracking of steel materials. Attached Figure Description

[0037] Figure 1(a)-Figure 1(e) The dataset contains images with different defects.

[0038] Figure 2 This is a flowchart of an industrial text image repair method based on feature recognition mechanism;

[0039] Figures 3(a)-3(e) To compare the restoration effects of different text image repair methods on the damaged image in Figure 1(c), Figure 3(a) shows the edge connection network, Figure 3(b) shows the gated convolutional network, Figure 3(c) shows the recurrent feature network, Figure 3(d) shows the method of this invention, and Figure 3(e) shows the real image data.

[0040] Figures 3(f)-3(j) To compare the restoration effects of different text image repair methods on the damaged image in Figure 1(d). Figure 3(f) shows the edge connection network, Figure 3(g) shows the gated convolutional network, Figure 3(h) shows the recurrent feature network, Figure 3(i) shows the method of this invention, and Figure 3(j) shows the real image data;

[0041] Figure 3(k)-Figure 3(o) To compare the restoration effects of different text image repair methods on the damaged image in Figure 1(e). Figure 3(k) shows the edge connection network, Figure 3(l) shows the gated convolutional network, Figure 3(m) shows the recurrent feature network, Figure 3(n) shows the method of this invention, and Figure 3(o) shows the real image data;

[0042] Figure 4 shows a comparison of the restoration effects of different methods on a public dataset. Figure 4(a) is the public dataset, Figure 4(b) is the processing method of the present invention, and Figure 4(c) is the real image data. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] This invention provides an industrial text image repair method based on a feature recognition mechanism. The method solves the data waste problem caused by the defective image in the slab number recognition system and fills the gap in research on text image repair in the industrial field.

[0045] Step 1: Obtain industrial text image data from the slab number recognition system and prepare a dataset.

[0046] Industrial text image data was obtained from the slab number recognition system. Images containing damaged areas were selected to create a test dataset. Intact images from the same system were overlaid with binary pixel masks to create a training dataset. It should be noted that the datasets can be allocated using the above method or generated through random fusion. The text region content and corresponding location information of the images within the datasets were labeled. Partially damaged image data is shown in Figure 1.

[0047] Step 2: Locate the defective region in the image to be repaired in the defective training dataset.

[0048] First, the image to be repaired in the dataset is traversed through three local convolutional layers to obtain information such as the location of defects, including the shape, size, location, and distance from the image boundary of any abnormal pixel regions. The feature value at position (x, y) in the i-th layer is multiplied pixel-by-pixel by the n-th convolutional kernel to obtain the updated mask value. Then, the difference matrix between the updated mask and the input mask is defined as the region to be inferred in the defective image. Finally, after processing through a fully connected network layer and an activation function, the difference matrix to be inferred is sent to the method described in step 3.

[0049] Step 3: Use the difference matrix obtained in Step 2 to perform feature recognition, narrowing down the defective region and filling in the feature value Vi. Repeat steps 2 and 3 to obtain the feature map group F;

[0050] The first feature recognition is performed on the difference matrix region. First, the difference matrix is ​​directly fed into the knowledge consistency attention module to directly calculate the feature value V of the mask region updated in step 2. i ={v i |i=1,2,3,…,d}; Then, repeat steps 2 and 3, and use the feature values ​​v calculated by the knowledge consistency attention module. i The feature map f is obtained by filling the difference matrix and then pooling. pool Continue until step 2 can no longer detect the missing area in the image, and record the resulting feature map group as...

[0051] Step 4: Perform a fusion operation on the feature map group F to obtain the feature map Fm, and then map Fm into a sequence of feature vectors through a two-layer fully connected convolutional layer.

[0052] The feature map group F is obtained by fusing the feature map group F. m F is achieved through two fully connected convolutional layers. m The mapping is a sequence of feature vectors, denoted as . Each feature vector Arranged horizontally from left to right at the original feature map position;

[0053] Step 5: Process the feature vector sequence obtained in Step 4 using a bidirectional long short-term memory network. The sequences are reordered to obtain a new sequence. Then, the attention mechanism is used to decode the new sequence twice to obtain the weight α and feature value Vh of each sequence element.

[0054] Using a bidirectional long short-term memory network module Calculation yields the new sequence H = {h} h The internal correlation degree R = {r | h = 1, 2, 3, ..., j} and H is {r} h |h=1,2,3,...,j}. The calculation process can be represented as: r h =BLSTM(r h-1 ,gh,h h The alignment factor α is obtained by scoring each element in H using an attention function and then normalizing the results. h G = {g} h =α h *h h |h=1,2,3,...,j} is the weighted sum of the new sequence H. After obtaining the internal correlation between H and H, the attention function is used to perform secondary decoding on H to obtain the feature value V of each sequence element h. h ;

[0055] Step 6: Apply weight α to the eigenvalues ​​V obtained in Step 5. h Perform secondary feature recognition, output the text content, and compare it with the text region information in the label described in step 1. Based on the comparison results, a reconstructed image after two feature recognitions can be obtained, or a reconstructed image obtained after 2n feature recognitions through n iterations of steps 2 to 6.

[0056] The eigenvalues ​​V obtained in step 5 are weighted according to weight α. h The output is text content, which is compared with the text region information in the label described in step 1. If they are equal, the feature map group F is used to adaptively fuse low-level high-resolution features and high-level semantic features to obtain the reconstructed image, where the adaptive fusion uses the FPN algorithm; if they are not equal, steps 2 to 6 are iterated n times to obtain the reconstructed image. The iteration continues until the loss function L is reached. totalIt is below the set threshold.

[0057] Step 7: Use the above repair method to repair the test set images to obtain reconstructed images. After re-applying the reconstructed images to the slab number recognition system, secondary slab number recognition can be achieved.

[0058] To verify the repair effect, this invention used four datasets for experiments. Three of the datasets are common image datasets used for text recognition tasks, and one dataset comes from a slab number recognition system. The three common image datasets are: IIIT5K-Words (IIIT), ICDAR2015 (IC15), and MSRA-td500.

[0059] The industrial text image repair method based on feature recognition mechanism proposed in this invention was compared with general image repair algorithms in the above dataset, and the repair results are shown in Figures 3 and 4.

[0060] To further validate the industrial text image inpainting method based on feature recognition mechanism proposed in this invention, the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) were used to quantitatively compare the models. The experimental results are as follows:

[0061]

[0062]

[0063] In summary, the industrial text image repair method based on feature recognition mechanism proposed in this invention can solve the problem of data waste caused by defective images in slab number recognition systems.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the scope defined by the claims of the present invention.

Claims

1. A method for industrial text image repair based on feature recognition, characterized in that, Includes the following steps: Step 1: Obtain industrial text image data from the slab number recognition system, and prepare defect training dataset and defect test dataset; add the content information and location information of the text region of the industrial text image data to the data label; Step 2: Locate the defective region in the image to be repaired in the defective training dataset; The defective region in the image to be repaired is obtained, and a difference matrix is ​​generated through calculation. Step 3: Perform feature identification using the difference matrix obtained in Step 2 to narrow down the defect area and fill in the feature value V. i Repeat steps 2 and 3 to obtain feature map group F; Step 4: Perform a fusion operation on the feature map group F to obtain feature map F. m F is achieved through two fully connected convolutional layers. m Mapped to a sequence of feature vectors ; Step 5: Process the feature vector sequence obtained in Step 4 using a bidirectional long short-term memory network. The sequences are reordered to obtain a new sequence. An attention mechanism is then used to decode the new sequence to obtain the weight α and feature value V for each element. h ; Step 6: Apply weight α to the eigenvalues ​​V obtained in Step 5. h Perform secondary feature recognition, output the text content, and compare it with the text region content information and location information in the data label described in step 1; based on the comparison results, obtain the reconstructed image after two feature recognitions, or obtain the reconstructed image after 2n feature recognitions through n iterations of steps 2 to 6; the reconstructed image is the final repaired industrial text image.

2. The industrial text image repair method based on feature recognition according to claim 1, characterized in that, Step 1 specifically involves: Industrial text image data is obtained from the slab number recognition system. Defective text images that cannot be recognized by the OCR system are collected, and binary pixel masks are input to overlay intact text images. The text region is a region different from the target repair region.

3. The industrial text image repair method based on feature recognition according to claim 2, characterized in that, Step 2 specifically involves: The image to be repaired in the dataset is traversed through g local convolutional layers to obtain information about the missing regions, including the shape, size, location, and holes at the boundary of the image to be repaired for abnormal pixels. The feature value of the abnormal pixel position (x, y) in the i-th local convolutional layer is multiplied pixel by pixel by the m-th convolutional kernel to obtain the updated mask value, represented as: ； The difference matrix between the updated mask and the input mask is defined as the defective region to be inferred in the defective image; finally, after processing by a fully connected network layer and an activation function, the processed difference matrix is ​​used for a first feature recognition.

4. The industrial text image repair method based on feature recognition according to claim 3, characterized in that, Step 3 specifically involves: The first feature recognition is performed on the difference matrix region; the difference matrix is ​​then input into the knowledge consistency attention module to calculate the feature values ​​of the mask region updated in step 2. Then, repeat steps 2 and 3, and use the feature values ​​v calculated by the knowledge consistency attention module. i The feature map f is obtained by filling the difference matrix and then pooling. pool Continue this process until no more missing areas are detected in step 2, then record the resulting feature map set as... .

5. The industrial text image repair method based on feature recognition according to claim 4, characterized in that, Step 4 specifically involves: The feature map group F is obtained by fusing the feature map group F. m F is achieved through two fully connected convolutional layers. m The mapping is a sequence of feature vectors, denoted as . ; Each feature vector In the original feature map F m They are arranged horizontally from left to right.

6. The industrial text image repair method based on feature recognition according to claim 1, characterized in that, Step 5 specifically involves: Using a bidirectional long short-term memory network module Calculation yields a new sequence. internal correlation with H ; The calculation process is expressed as follows: The alignment factor is obtained by scoring each element in the new sequence H using an attention function and then normalizing the results. The weighted sum of the new sequence H; After obtaining the new sequence H and its internal correlation, the attention function is used to perform secondary decoding on the new sequence H to obtain the feature value V of each sequence element h. h .

7. The industrial text image repair method based on feature recognition according to claim 6, characterized in that, Step 6 specifically involves: The eigenvalues ​​V obtained in step 5 are weighted according to weight α. h The output is text content, which is compared with the text region content information and text information in the data labels described in step 1. If they are equal, the low-level high-resolution features and high-level semantic features of feature map group F are adaptively fused to obtain the reconstructed image, where the adaptive fusion uses the FPN algorithm; if they are not equal, steps 2 to 6 are iterated n times to obtain the reconstructed image; the iteration continues until the loss function is reached. Below the set threshold, the loss function required for iteration It is expressed as follows: ； Where λ is a learnable parameter, and style loss is... Perceived loss and predicted loss They are represented as follows: ； in, The actual value in the label of step 1. A vector representing all parameters in the feature recognition mechanism. Feature vector sequence, The i-th feature map after pooling is represented by H, W, and C, which represent the length, width, and number of channels of the input image, respectively.

Images