Bill normativity inspection AI identification method

Abstract

The invention discloses a bill normativity inspection AI identification method, and relates to the technical field of electric power bill identification, in particular to a bill normativity inspection AI identification method, which comprises the following steps of: carrying out overlapping region combination, shading interference removal, character color highlighting and binarization processing on identified information regions; analyzing and translating the detected shape by adopting a character recognition method; the method comprises the following specific steps: S1, inputting bill data; s2, performing layout analysis on the preprocessed image; s3, character recognition operation is carried out; s4, restoring the layout; and S5, post-processing and proofreading. According to the bill normalization inspection AI identification method, the OCR automatic identification technology is adopted, and the functions of intelligent identification, intelligent query, intelligent numbering and the like of work tickets are achieved; and technologies such as error mode and program frequency spectrum auxiliary defect positioning are comprehensively used to realize automatic error correction of batch work ticket contents, so that the labor efficiency of operation and maintenance personnel is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power bill recognition, and specifically to an AI recognition method for checking the standardization of bills. Background Art

[0002] During the production process of power enterprises, the work ticket and operation ticket systems have been effective work systems that have been consistently adhered to for many years and are the main content of the "Power Safety Work Regulations"; a work ticket is a written order for working on electrical equipment and is a written basis for implementing work safety measures, including the first type of electrical work ticket, the second type of electrical work ticket, the thermal and mechanical work ticket, the instrument control work ticket, etc.; an operation ticket is a written basis for operation tasks, operation procedures, and operation activities and is an important organizational measure to ensure the safe operation of electrical equipment during operation; when issuing a work ticket, it is necessary to check whether the safety measures filled in on the work ticket are correct and complete; when issuing an operation ticket, it is necessary to review whether the operation items filled in for this operation task are complete and whether the sequence relationship between the operation items is correct.

[0003] The traditional method of checking work tickets has the following problems:

[0004] (1) There are many types of operation tickets with different styles and significant differences in semantic expressions, and there may be large errors in the scanning and recognition results.

[0005] (2) There are many handwritten fonts in power operation tickets, including the signatures of the issuer and the recipient, time, operation items, etc. The writing styles of handwritten fonts vary from person to person, and the font structures are complex and diverse, increasing the difficulty of text recognition of power operation tickets and not well meeting people's usage requirements. Summary of the Invention

[0006] In view of the deficiencies of the prior art, the present invention provides an AI recognition method for checking the standardization of bills, which solves the problems proposed in the above background art that there are many handwritten fonts in power operation tickets, increasing the difficulty of text recognition of power operation tickets, and there may be large errors in the scanning and recognition results.

[0007] To achieve the above objectives, the present invention is realized through the following technical solutions: An AI recognition method for checking the standardization of bills includes merging overlapping regions of the information area to be recognized, removing shading interference, highlighting character colors, and performing binarization processing, and using a character recognition method to analyze and translate the detected shapes.

[0008] The specific steps are as follows:

[0009] S1. After inputting the bill data, perform preprocessing operations on the work ticket image. Use an identifier to convert the paper work ticket into a picture, and then perform operations such as removing black edges, removing noise points, correcting skew, and grayscaling on the work ticket picture.

[0010] S2. Perform layout analysis on the preprocessed image, use a suitable cutting model to divide the document into paragraphs and cut the connected characters to obtain materials for character recognition, and use the information area feature database constructed by pre-training and self-learning through information areas to identify the information area of the preprocessed work ticket image;

[0011] S3. Perform character recognition operations. Character recognition uses the feature extraction method to perform template matching between the materials obtained from layout analysis and the database dictionary;

[0012] S4. Layout restoration. Layout restoration is to typeset the recognized materials according to the input material format to ensure maximum restoration in terms of paragraphs, positions, etc.;

[0013] S5. Post-processing and proofreading. In the post-processing and proofreading process, adjust and correct the recognized results according to the text context, logical relationship or other auxiliary information, and output the required document format to obtain the layout information output.

[0014] Optionally, the specific operation steps of the preprocessing operation in S1 are as follows:

[0015] (1) Grayscale processing: Convert the color picture into a grayscale picture;

[0016] (2) Binarization: Convert the grayscale picture into a black and white picture;

[0017] (3) Denoising: Eliminate the noise points on the black and white picture to make the picture look cleaner;

[0018] (4) Rotation: Rotate the picture clockwise and counterclockwise to find an optimal horizontal position.

[0019] Optionally, both the character cutting and layout cutting in S2 include horizontal cutting and vertical cutting.

[0020] Optionally, the layout analysis in S2 also includes boundary and shading processing, merging overlapping areas of the recognized information area, removing shading interference, highlighting character colors, and binarization processing.

[0021] Optionally, the layout analysis in S2 includes character cutting and layout cutting respectively.

[0022] Optionally, the horizontal cutting in S2 is to cut the picture with the adjusted horizontal position row by row; the vertical cutting is to cut the row-by-row pictures column by column to produce individual characters.

[0023] Optionally, in S3, an optical character recognition technology is used for the character recognition operation to analyze, translate, and process the detected shape, and retrieve the corresponding database, so as to obtain computer text and layout information that can be edited by the user.

[0024] Optionally, the optical character recognition technology is specifically the CBRT recognition technology. The specific process of the CBRT recognition technology is as follows:

[0025] (1) Using the sample pictures as the training data set, construct a custom three-layer convolutional network model and train to output a non-linear mapping function;

[0026] (2) Using the test pictures as the input of the non-linear mapping function to obtain test pictures with high PSNR values;

[0027] (3) Using the sample pictures as the training data set, construct an integrated CNN model based on imaginary strokes, path signatures, and eight-direction features, and train to obtain a classification model;

[0028] (4) Using the test pictures as the input of the classification model, calculate the classification results using the simple average method.

[0029] The present invention provides a method for AI recognition of bill standardization inspection, which has the following beneficial effects:

[0030] 1. For this method for AI recognition of bill standardization inspection, by comprehensively using error patterns and program spectra to assist in defect location, and on this basis, using the defect code automatic generation technology based on deep learning and the repair verification method based on program synthesis technology to achieve automatic correction.

[0031] 2. For this method for AI recognition of bill standardization inspection, perform program analysis on the defective program to obtain the abstract syntax tree of the defective program, and on this basis, comprehensively use defect patterns and program spectra to assist in defect location to obtain potential defect locations in the defective program.

[0032] 3. For this method for AI recognition of bill standardization inspection, use a long short-term neural network to learn the statement structure in the correct program, form a structure model through training, at the potential defect locations in the incorrect program, use the trained structure model to predict the statement structure at the defect locations, and expand this structure to obtain all possible defect repair candidates.

[0033] 4. For this method for AI recognition of bill standardization inspection, replace the defective statement with the repair candidates in the form of a selection expression, and use the example program as a specification. Use the program synthesis technology to select appropriate repair options for each selection expression to meet the specification, and finally replace the corresponding defective statement with the obtained repair options to generate the repaired correct program.

[0034] 5. The AI recognition method for the standardization inspection of the bill adopts OCR automatic recognition technology to realize functions such as intelligent recognition, intelligent query, and intelligent numbering of work tickets; research the automatic defect code generation technology based on deep learning and the repair verification method based on program synthesis technology, and comprehensively use technologies such as error patterns and program spectra to assist defect location to realize the error correction of automatic batch work ticket content, greatly improving the labor efficiency of operation and maintenance personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the OCR technical process in the AI recognition method for the standardization inspection of the bill;

[0036] Figure 2 It is a schematic diagram of the CBRT operation process in the AI recognition method for the standardization inspection of the bill;

[0037] Figure 3 It is a schematic diagram of the convolutional neural network model in the AI recognition method for the standardization inspection of the bill;

[0038] Figure 4 It is a schematic diagram of establishing a similarity model by Euclidean distance in the AI recognition method for the standardization inspection of the bill. SPECIFIC EMBODIMENTS

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0040] In the description of the present invention, unless otherwise stated, the meaning of "a plurality" is two or more; the orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "inner", "outer", "front end", "backend", "head", "tail", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, terms such as "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0041] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0042] Embodiment 1

[0043] Please refer to Figures 1 to 4 , the present invention provides a technical solution: a method for AI recognition of bill standardization inspection, including merging overlapping regions of the recognized information region, removing shading interference, highlighting character colors, and binarization processing, and using a character recognition method to analyze and translate the detected shapes;

[0044] The specific steps are as follows:

[0045] S1. After the bill data is input, preprocessing operations are performed on the working bill image. The paper-based work ticket is converted into a picture using an identifier, and then the work ticket picture is processed such as removing black edges, removing noise points, rectifying, and grayscaling;

[0046] S2. Perform layout analysis on the image after preprocessing. Use a suitable cutting model to divide the document into paragraphs and cut the connected characters to obtain the materials for character recognition. Utilize the information region feature database constructed by pre-training and self-learning through the information region to recognize the information region of the preprocessed working bill image;

[0047] S3. Perform character recognition operations. Character recognition uses the feature extraction method to perform template matching between the materials obtained from the layout analysis and the database dictionary;

[0048] S4. Layout restoration, that is, typesetting the recognized materials according to the input material format to ensure maximum restoration in terms of paragraphs, positions, etc.;

[0049] S5. Post-processing and proofreading. In the post-processing and proofreading process, adjust and correct the recognized results according to the text context, logical relationship, or other auxiliary information, and output the required document format to obtain the layout information output.

[0050] In this embodiment, as Figure 1 shown, the specific operation steps of the preprocessing operation in S1 are as follows:

[0051] (1) Grayscale processing: Convert the color picture into a grayscale picture;

[0052] (2) Binarization: Convert the grayscale picture into a black and white picture;

[0053] (3) Denoising: Eliminate the noise points on the black and white picture to make the picture look cleaner;

[0054] (4) Rotation: Rotate the picture clockwise and counterclockwise to find an optimal horizontal position.

[0055] In this embodiment, as Figure 1 shown, both the character cutting and layout cutting in S2 include horizontal cutting and vertical cutting.

[0056] In this embodiment, as Figure 1 shown, the layout analysis in S2 also includes border and shading processing, merging overlapping regions of the identified information regions, removing shading interference, highlighting character colors, and binarization processing.

[0057] In this embodiment, as Figure 1 shown, the layout analysis in S2 includes character cutting and layout cutting respectively.

[0058] In this embodiment, as Figure 1 shown, the horizontal cutting in S2 is to cut the picture with adjusted horizontal position row by row; the vertical cutting is to cut the row-by-row pictures column by column to produce individual characters.

[0059] In this embodiment, as Figure 1 shown, in S3, the character recognition operation uses optical character recognition technology to analyze, translate and process the detected shapes, and retrieves the corresponding database to obtain computer text and layout information that can be edited by the user.

[0060] In this embodiment, as Figure 1 shown, the optical character recognition technology is specifically the CBRT recognition technology, and the specific process of the CBRT recognition technology is as follows:

[0061] (1) Using the sample pictures as the training data set, constructing a custom three-layer convolutional network model, and training to output a non-linear mapping function;

[0062] (2) Using the test pictures as the input of the non-linear mapping function to obtain test pictures with high PSNR values;

[0063] (3) Using the sample pictures as the training data set, constructing an integrated CNN model based on imaginary strokes, path signatures and eight-direction features, and training to obtain a classification model;

[0064] (4) Using the test pictures as the input of the classification model, and calculating the classification results using the simple average method.

[0065] Embodiment 2

[0066] Please refer to Figures 1 to 4 , the present invention provides a technical solution: a method for AI recognition of bill standardization inspection, including merging overlapping regions of the identified information regions, removing shading interference, highlighting character colors, and binarization processing, and using a character recognition method to analyze, translate and process the detected shapes;

[0067] The specific steps are as follows:

[0068] S1. After the bill data is input, the work bill image is pre-processed, and the paper work bill is converted into a picture using a recognition device, and then the work bill picture is processed such as black edge removal, noise removal, de-skew correction and grayscale conversion;

[0069] The specific steps of the preprocessing operation are as follows:

[0070] (1) Grayscale processing: converting color images into grayscale images;

[0071] (2) Binarization: converting grayscale images into black and white images;

[0072] (3) Denoising: Remove noise from black and white images to make them look cleaner;

[0073] (4) Rotation: Rotate the image clockwise and counterclockwise to find the best horizontal position;

[0074] S2. Perform layout analysis on the pre-processed image, divide the document into paragraphs and cut the contiguous characters using a suitable cutting model, obtain material for character recognition, and use the information region feature database constructed by self-learning through information region training to perform information region recognition on the pre-processed work bill image; perform boundary and shading processing, including merging overlapping regions of the recognized information regions, removing shading interference, highlighting character colors, and binarization processing;

[0075] The layout analysis includes character cutting and layout cutting. Both character cutting and layout cutting include horizontal cutting and vertical cutting. Horizontal cutting is to cut the pictures with adjusted horizontal positions row by row. Vertical cutting is to cut the pictures row by row column by column to produce individual characters.

[0076] S3, performing character recognition operation, character recognition uses feature extraction method to match the material obtained by layout analysis with the database dictionary for template matching, character recognition operation uses optical character recognition technology to analyze and translate the detected shape, and call the corresponding database to obtain computer text and layout information that can be edited by the user;

[0077] Optical character recognition technology is specifically CBRT recognition technology. The specific process of CBRT recognition technology is as follows:

[0078] (1) The sample images are used as training data sets to build a custom three-layer convolutional network model and train the output nonlinear mapping function;

[0079] (2) The test image is used as an input to the nonlinear mapping function to obtain a test image with a high PSNR value;

[0080] (3) Using the sample images as the training data set, an integrated CNN model based on imaginary strokes, path signatures, and eight-direction features is constructed, and a classification model is obtained through training;

[0081] (4) Using the test images as the input of the classification model, the classification results are calculated using the simple average method;

[0082] S4. Layout restoration. Layout restoration means typesetting the recognized materials according to the input material format to ensure maximum restoration in terms of paragraphs, positions, etc.;

[0083] S5. Post-processing and proofreading. In the post-processing and proofreading process, the recognized results are adjusted and corrected according to the text context, logical relationship, or other auxiliary information, and the required document format is output to obtain the layout information output.

[0084] Regarding the characteristics of handwritten fonts, a convolutional neural network is selected to be constructed by imitating the visual perception mechanism of organisms. After a certain amount of training, the convolutional neural network can automatically complete the step of extracting features from images, and can reduce the recognition misjudgment caused by factors such as image translation, rotation, stretching, and partial occlusion;

[0085] The convolutional neural network imitates the visual perception of organisms. It can directly input the original image. Through weight sharing, the number of free parameters in the network is reduced, greatly reducing the complexity of the network model. It not only has the characteristics of traditional neural networks such as self-adaptation, but also has characteristics such as automatic feature extraction; The convolutional neural network is composed of a convolutional layer, a pooling layer, and a fully connected layer. Among them, the convolutional layer and the pooling layer play the role of automatically extracting the features of the image, and the structure of the fully connected layer is similar to that of the BP neural network;

[0086] The recognition process using the convolutional neural network is as follows:

[0087] First, directly input the image data and the corresponding labels into the network model without specifying the image feature extraction method; then, use the backpropagation algorithm to automatically adjust the model parameters according to the labels and the predicted values of the model, and extract suitable image features as the classification basis; The convolutional neural network not only reduces the operation difficulty of feature extraction, but also avoids the errors caused by human factors in the feature selection process;

[0088] Inputting the image into the trained model, the probability of the image belonging to each category can be obtained; when the image is damaged or polluted, the probability of belonging to the correct category will decrease, but it can still be recognized because the convolutional network pays more attention to the generalization features among the same categories; The development of the convolutional neural network has brought the classification and recognition technology into the stage of automatic feature extraction and classification and recognition. However, the convolutional neural network has high requirements for the number of samples, and the computational amount of training the model far exceeds the manual feature extraction technology;

[0089] In addition, the recognition effect of a convolutional neural network is highly related to its depth. The deeper the depth, the better the recognition effect, but it is also more prone to overfitting (i.e., the recognition accuracy for the training dataset is high, while the recognition accuracy for the test dataset is low). If the network depth is too shallow, it is prone to underfitting (i.e., the recognition accuracy for both the training dataset and the test dataset is low).

[0090] Based on the UCI optical character dataset, a feature value data model is built, and the information in the scanned materials is processed based on the above data model. The specific process is as follows: First, in combination with the actual requirements in the application process, the title row is used as the feature value to classify the feature recognition characters in multiple work items, realizing the modularization of the work content. Then, the similarity between characters' features is found, and the average value of each feature of each character is taken as the benchmark for character discrimination. To improve the accuracy of character recognition, a common and intuitive similarity algorithm - Euclidean distance - is used to establish a similarity model.

[0091] The Euclidean distance similarity model takes the items commonly evaluated between characters as dimensions to establish a multi-dimensional space. By locating the score of a character in different dimensions in this multi-dimensional space, the Euclidean distance between any two positions reflects to a certain extent the similarity between the two characters. To achieve the matching of feature values on the string and thus ensure the smoothness of the entire paragraph, the neural network algorithm is applied, enabling it to archive the historical data recognized by users to form a usable database for optimizing the recognition speed of subsequent input characters and strings. This model streamlines the network layers to improve the model training efficiency. At the same time, multiple handwriting features are introduced to replace the original image input, overcoming the limitation of CNN in learning the spatial features of the original image and enhancing the accuracy of handwritten font recognition.

[0092] The above is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, makes equivalent substitutions or changes, and all should be covered within the protection scope of the present invention.

Claims

1. AI recognition method for bill standardization inspection, Characterized in that: It includes merging overlapping regions, removing shading interference, highlighting character colors, and binarization processing on the recognized information region, and using character recognition methods to analyze and translate the detected shapes; The specific steps are as follows: S1. After the bill data is input, preprocessing operations are performed on the working bill image. The paper-based work ticket is converted into a picture using an identifier, and then the work ticket picture is processed such as removing black edges, removing noise points, rectifying deviation, and grayscale processing; S2. Perform layout analysis on the image after preprocessing. Use a suitable cutting model to divide the document into paragraphs and cut the connected characters to obtain the materials for character recognition. Utilize the information region feature database constructed by pre-training and self-learning through the information region to recognize the information region of the preprocessed working bill image; S3. Perform character recognition operations. Character recognition uses the feature extraction method to perform template matching between the materials obtained from layout analysis and the database dictionary; S4. Layout restoration. Layout restoration means typesetting the recognized materials according to the input material format to ensure maximum restoration in terms of paragraphs, positions, etc.; S5. Post-processing and proofreading. During post-processing and proofreading, the recognized results are adjusted and corrected according to the text context, logical relationship, or other auxiliary information, and the required document format is output to obtain the layout information output.

2. The AI recognition method for bill standardization inspection according to claim 1, Characterized in that: The specific operation steps of the preprocessing operation in S1 are as follows: (1) Grayscale processing: Convert the color picture into a grayscale picture; (2) Binarization: Convert the grayscale picture into a black and white picture; (3) Denoising: Eliminate the noise points on the black and white picture to make the picture look cleaner; (4) Rotation: Rotate the picture clockwise and counterclockwise to find an optimal horizontal position.

3. The AI recognition method for bill standardization inspection according to claim 1, Characterized in that: Both character cutting and layout cutting in S2 include horizontal cutting and vertical cutting.

4. The AI recognition method for bill standardization inspection according to claim 1, Characterized in that: The layout analysis in S2 also includes boundary and shading processing, merging overlapping regions, removing shading interference, highlighting character colors, and binarization processing on the recognized information region.

5. The AI recognition method for bill standardization inspection according to claim 1, Characterized in that: The layout analysis in S2 includes character cutting and layout cutting respectively.

6. The AI recognition method for bill standardization inspection according to claim 3, Characterized in that: The horizontal cutting in S2 is to cut the picture with the adjusted horizontal position row by row; the vertical cutting is to cut the row-by-row pictures column by column to produce single characters.

7. The AI recognition method for bill standardization inspection according to claim 1, Characterized in that: In the character recognition operation in S3, the optical character recognition technology is adopted to analyze and translate the detected shape, and the corresponding database is retrieved, so as to obtain the computer text and layout information that can be edited by the user.

8. The AI recognition method for bill standardization inspection according to claim 7, characterized in that: the optical character recognition technology is specifically the CBRT recognition technology, and the specific process of the CBRT recognition technology is as follows: (1) Using the sample pictures as the training data set, constructing a custom three-layer convolutional network model, and training to output a non-linear mapping function; (2) Using the test pictures as the input of the non-linear mapping function to obtain test pictures with high PSNR values; (3) Using the sample pictures as the training data set, constructing an integrated CNN model based on imaginary strokes, path signatures and eight-direction features, and training to obtain a classification model; (4) Using the test pictures as the input of the classification model, and calculating the classification results using the simple average method.

Images