Field identification method and device, computer device and computer readable storage medium

Abstract

The application discloses a field recognition method and device, computer equipment and a computer readable storage medium, relates to the computer vision technology and the financial field, and does not need to manually extract features. The model can learn feature representation autonomously according to a large amount of sample data, and adapt to the recognition demand of multiple field images. The method comprises the following steps: acquiring an original image set, performing data labeling on each original image in the original image set to obtain a labeled image set, cutting each labeled image in the labeled image set according to a preset cutting standard to obtain a sample data set, and performing model training on an initial field recognition model by using the sample data set to obtain a field recognition application model; in response to a field recognition instruction, acquiring a to-be-recognized image carried in the field recognition instruction, inputting the to-be-recognized image into the field recognition application model for recognition, and obtaining a target field position and a target field category corresponding to each target text field existing in the to-be-recognized image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and finance, and particularly relates to a field recognition method and device, a computer device and a computer readable storage medium. BACKGROUND

[0002] With the development of computer vision technology, an OCR (Optical Character Recognition) project appears. The OCR project first detects text on a picture by using a detection model to obtain text line coordinates, then crops a text line picture according to the coordinates and identifies the text by using a recognition model. After obtaining the detection text line coordinates and text content, category recognition is performed according to the coordinate positions and text features. However, the traditional OCR method usually relies on manually designed features and rules for classification and correction in the post-processing process. Such a method has poor generalization ability and low recognition accuracy, and is difficult to process complex multi-text images. Therefore, there is an urgent need for a field recognition method suitable for complex text. SUMMARY

[0003] In view of this, the present application provides a field recognition method and device, a computer device and a computer readable storage medium, and the main purpose is to solve the problem that the traditional OCR method usually relies on manually designed features and rules for classification and correction in the post-processing process. Such a method has poor generalization ability and low recognition accuracy, and is difficult to process complex multi-text images.

[0004] According to a first aspect of the present application, a field recognition method is provided, which comprises:

[0005] An original image set is obtained, and each original image in the original image set is data-labeled to obtain a labeled image set, wherein each original image contains a plurality of text fields, and the data labeling is used to label the field position and field category of each text field;

[0006] Each labeled image in the labeled image set is cropped according to a preset cropping standard to obtain a sample data set, and an initial field recognition model is trained by using the sample data set to obtain a field recognition application model, wherein the field recognition application model is a full convolutional neural network;

[0007] In response to a field recognition instruction, a to-be-recognized image carried in the field recognition instruction is obtained, and the to-be-recognized image is input into the field recognition application model for recognition to obtain a target field position and a target field category corresponding to each target text field existing in the to-be-recognized image.

[0008] Optionally, the inputting the to-be-recognized image into the field recognition application model for recognition to obtain a target field position and a target field category corresponding to each target text field existing in the to-be-recognized image comprises:

[0009] performing feature extraction on the to-be-recognized image by using a convolutional layer of the field recognition application model to obtain a target feature map, and performing image processing on the target feature map by using a classification layer of the field recognition application model to obtain a segmentation tensor and a classification tensor, the convolutional layer being a feature pyramid network structure;

[0010] reading a pixel value of each pixel in the segmentation tensor at an output layer of the field recognition application model, marking a specified pixel with a pixel value greater than a preset pixel threshold as a foreground region, marking other pixels as a background region, and calculating a continuous boundary in the foreground region based on an image processing algorithm to obtain a plurality of text boxes of a plurality of target text fields, taking a position of a text box of each target text field as a target field position of the target text field, and for each text box in the plurality of text boxes, determining all text pixels corresponding to the text box, querying a field category of each text pixel in the classification tensor, and taking a field category with the most pixel values as a target field category of a target text field corresponding to the text box, the other pixels being used to indicate pixels other than the specified pixels in the segmentation tensor.

[0011] Optionally, the classification layer comprises a classification tensor output layer and a segmentation tensor output layer, a number of channels of the classification tensor output layer being equal to a number of categories of field categories, the classification tensor output layer being used to determine a field category to which a pixel value belongs, and a number of channels of the segmentation tensor output layer being 1, the segmentation tensor output layer being used to determine a coordinate of a text field.

[0012] Optionally, the obtaining an original image set comprises:

[0013] performing image preprocessing on each original image, the image preprocessing comprising but not limited to noise removal processing, size unification processing and contrast enhancement processing;

[0014] using a preset annotation tool to mark a field position of a plurality of text fields existing in each original image, and setting a category label for each text field, associating the category label of each text field with a corresponding field position to obtain an annotated image;

[0015] annotating each original image in the original image set to obtain the annotated image set.

[0016] Optionally, the cropping each of the labeled images in the labeled image set according to the preset cropping standard to obtain the sample data set comprises:

[0017] reading a plurality of cropping sizes corresponding to the preset cropping standard;

[0018] For each of the labeled images in the labeled image set, querying the field positions of a plurality of text fields present in the labeled image, and determining a plurality of text boxes corresponding to the plurality of text fields according to the field positions;

[0019] determining the text box of each of the text fields in the labeled image, continuously selecting any of the plurality of cropping sizes for cropping according to a cropping condition, and scaling a plurality of cropped boxes obtained by cropping to a plurality of background boxes with consistent size, the cropping condition being not cutting off the text box;

[0020] taking the plurality of text boxes and the plurality of background boxes corresponding to the labeled image as sample data corresponding to the labeled image;

[0021] determining the sample data corresponding to each of the labeled images to obtain the sample data set.

[0022] Optionally, after the determining the sample data corresponding to each of the labeled images to obtain the sample data set, the method further comprises:

[0023] numbering each of the sample data in the sample data set, extracting a specified sample data corresponding to a specified number to obtain a plurality of specified sample data, the specified number being any value, and the number of the plurality of specified sample data satisfying a preset splitting value;

[0024] aggregating the plurality of specified sample data into a training sample set, and aggregating the remaining sample data after extraction into a test sample set.

[0025] Optionally, the model training of the initial field recognition model using the sample data set to obtain the field recognition application model comprises:

[0026] inputting the training sample set in the sample data set into the initial field recognition model for field position recognition and field type recognition, updating the model parameters of the initial field recognition model according to the recognition result to obtain a to-be-tested field recognition model;

[0027] inputting the test sample set in the sample data set into the to-be-tested field recognition model for model testing to obtain a test result;

[0028] when the accuracy of the test result meets a preset accuracy threshold, obtaining the field recognition application model.

[0029] According to a second aspect of the present application, a field recognition device is provided, which comprises:

[0030] An acquisition module is configured to acquire an original image set, perform data labeling on each original image in the original image set to obtain a labeled image set, wherein a plurality of text fields exist in each original image, and the data labeling is configured to label a field position and a field category of each text field;

[0031] A cropping module is configured to crop each labeled image in the labeled image set according to a preset cropping standard to obtain a sample data set, and perform model training on an initial field recognition model using the sample data set to obtain a field recognition application model, wherein the field recognition application model is a full convolutional neural network.

[0032] An identification module is configured to, in response to a field recognition instruction, acquire a to-be-recognized image carried in the field recognition instruction, and input the to-be-recognized image into the field recognition application model for recognition to obtain a target field position and a target field category corresponding to each target text field existing in the to-be-recognized image.

[0033] Optionally, the identification module is configured to perform feature extraction on the to-be-recognized image using a convolutional layer of the field recognition application model to obtain a target feature map, and perform image processing on the target feature map at a classification layer of the field recognition application model to obtain a segmentation tensor and a classification tensor, wherein the convolutional layer is a feature pyramid network structure; read a pixel value of each pixel in the segmentation tensor at an output layer of the field recognition application model, mark a specified pixel with a pixel value greater than a preset pixel threshold as a foreground region, mark other pixels as a background region, and calculate a continuous boundary in the foreground region based on an image processing algorithm to obtain a plurality of text boxes of a plurality of target text fields, take a position of a text box of each target text field as a target field position of the target text field, and for each text box in the plurality of text boxes, determine all text pixels corresponding to the text box, query a field category of each text pixel in the classification tensor, and take a field category with the most pixel values as a target field category of a target text field corresponding to the text box, wherein the other pixels are used to indicate pixels other than the specified pixels in the segmentation tensor.

[0034] Optionally, the classification layer comprises a classification tensor output layer and a segmentation tensor output layer, wherein a number of channels of the classification tensor output layer is equal to a number of categories of field categories, and is used to determine a field category to which a pixel value belongs, and a number of channels of the segmentation tensor output layer is 1, and is used to determine a coordinate of a text field.

[0035] Optionally, the obtaining module is configured to: obtain each of the original images to perform image preprocessing, the image preprocessing including but not limited to noise removal processing, size unification processing, and contrast enhancement processing; use a preset labeling tool to label field positions of a plurality of text fields existing in each of the original images, set a category label for each of the text fields, associate the category label of each of the text fields with the corresponding field position, and obtain a labeled image; and label each of the original images in the original image set to obtain the labeled image set.

[0036] Optionally, the cropping module is configured to: read a plurality of cropping sizes corresponding to the preset cropping standard; for each of the labeled images in the labeled image set, query field positions of a plurality of text fields existing in the labeled image, and determine a plurality of text boxes corresponding to the plurality of text fields according to the field positions; determine a text box of each of the text fields in the labeled image, continuously select any of the plurality of cropping sizes to perform cropping according to a cropping condition, and scale a plurality of cropped boxes obtained by cropping into a plurality of background boxes with consistent size, the cropping condition being that the text box is not cut off; take the plurality of text boxes and the plurality of background boxes corresponding to the labeled image as sample data corresponding to the labeled image; determine the sample data corresponding to each of the labeled images to obtain the sample data set.

[0037] Optionally, the cropping module is further configured to: number each of the sample data in the sample data set, extract a specified sample data corresponding to a specified number, obtain a plurality of specified sample data, the specified number being any value, and the number of the plurality of specified sample data satisfying a preset split value; aggregate the plurality of specified sample data into a training sample set, and aggregate the sample data remaining after extraction into a test sample set.

[0038] Optionally, the cropping module is configured to: input the training sample set in the sample data set into the initial field recognition model to perform field position recognition and field type recognition, update model parameters of the initial field recognition model according to a recognition result, and obtain a to-be-tested field recognition model; input the test sample set in the sample data set into the to-be-tested field recognition model to perform model testing, and obtain a test result; and when an accuracy of the test result satisfies a preset accuracy threshold, obtain the field recognition application model.

[0039] According to the third aspect of the present application, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method of any one of the first aspect when executing the computer program.

[0040] According to the fourth aspect of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the method according to any one of the first aspect.

[0041] By means of the technical solutions described above, the field recognition method, device, computer equipment and computer readable storage medium provided by the present application are provided. The present application first acquires an original image set. Each original image in the original image set has a plurality of text fields. The field position and field category of each text field in each original image are labeled to obtain a labeled image set. Further, each labeled image in the labeled image set is cropped according to a preset cropping standard to obtain a sample data set. Next, the initial field recognition model of the full convolutional neural network is trained by using the sample data set to obtain a field recognition application model. Finally, in response to a field recognition instruction, the image to be recognized carried in the field recognition instruction is acquired, and the image to be recognized is input into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the image to be recognized. In the embodiment of the present application, the full convolutional neural network can extract the features of the image through multi-layer convolution and pooling operations. This enables the network to simultaneously meet the needs of large and small texts, enabling the network to take into account text fields of different sizes and improving the accuracy of field recognition. In addition, the full convolutional neural network can automatically learn the feature representation of the field position and category through supervised training. Compared with traditional methods, manual feature extraction is no longer needed, and the model can automatically learn more expressive feature representations from a large amount of sample data. This end-to-end learning method has better generalization ability and can adapt to different scene and text style recognition requirements.

[0042] The above description is only a summary of the technical solutions of the present application. In order to enable the technical means of the present application to be more clearly understood, and to be implemented according to the content of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more apparent and easy to understand, the following detailed description of the specific embodiments of the present application is provided. BRIEF DESCRIPTION OF DRAWINGS

[0043] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the accompanying drawings indicate the same or similar components. In the drawings:

[0044] Figure 1 A field recognition method flowchart provided by an embodiment of the present application is shown;

[0045] Figure 2AA field recognition method provided by an embodiment of the present application is shown in a flowchart.

[0046] Figure 2B A recognition process of a field recognition method provided by an embodiment of the present application is shown in a flowchart.

[0047] Figure 3 A structure of a field recognition device provided by an embodiment of the present application is shown in a flowchart.

[0048] Figure 4 A device structure of a computer device provided by an embodiment of the present application is shown in a flowchart. DETAILED DESCRIPTION

[0049] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood and so that the scope of the present application can be accurately conveyed to those skilled in the art.

[0050] An embodiment of the present application provides a field recognition method, as shown in the method includes: Figure 1

[0051] 101, obtaining an original image set, performing data labeling on each original image in the original image set to obtain a labeled image set, wherein a plurality of text fields exist in each original image, and the data labeling is used for labeling a field position and a field category of each text field.

[0052] ​With the development of computer vision technology, an OCR (Optical Character Recognition) project appears. The OCR project first detects the text line coordinates on the picture by using a detection model, then crops the text line picture according to the coordinates and recognizes the text by using a recognition model. After obtaining the detection text line coordinates and text content, the category recognition is performed according to the coordinate position and text features. However, the traditional OCR method usually relies on manually designed features and rules for classification and correction in the post-processing process. Such method has poor generalization ability and low recognition accuracy, and is difficult to process complex multi-text images. Therefore, there is an urgent need for a field recognition method suitable for complex text. Therefore, the present application provides a field recognition method. First, an original image set is obtained. Each original image in the original image set has multiple text fields. The field position and field category of each text field in each original image are labeled to obtain a labeled image set. Further, each labeled image in the labeled image set is cropped according to a preset cropping standard to obtain a sample data set. Next, the initial field recognition model of the full convolutional neural network is trained by using the sample data set to obtain a field recognition application model. Finally, in response to a field recognition instruction, a to-be-recognized image carried in the field recognition instruction is obtained, and the to-be-recognized image is input into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the to-be-recognized image. In the embodiment of the present application, the full convolutional neural network can extract the features of the image through multi-layer convolution and pooling operations. This enables the network to simultaneously meet the needs of large and small texts, allowing the network to accommodate text fields of different sizes and improving the accuracy of field recognition. In addition, the full convolutional neural network can automatically learn the feature representation of the field position and category through supervised training. Compared with the traditional method, the feature representation can be learned more expressively by the model according to a large amount of sample data. Such end-to-end learning method has better generalization ability and can adapt to the recognition needs of different scenes and text styles.

[0053] In the embodiment of the present application, the relevant staff first collects a batch of original images with multiple text fields and uploads them to the field recognition system. After obtaining the original images, the field recognition system stores the original images in the original image set. It should be noted that these original images with multiple text fields can be invoices and other proof documents. In actual operation, the relevant staff can search for invoice images on the Internet to collect original images online, or collect original images from customers and partners offline, to ensure that the original image set contains invoice images of different types, layouts, and formats. Further, the field recognition system uses an image labeling tool preset by the relevant staff to label the field position and field category of each text field present in each original image, obtaining a labeled image set.

[0054] 102. According to the preset cropping standard, each labeled image in the labeled image set is cropped to obtain a sample data set, and the initial field recognition model is trained using the sample data set to obtain a field recognition application model, and the field recognition application model is a fully convolutional neural network.

[0055] In the embodiment of the present application, for each labeled image, the field recognition system needs to perform a cropping operation according to the determined field region in the labeled image. Specifically, the text box corresponding to each text field in the labeled image is determined, and the labeled image is cropped according to the preset cropping standard, avoiding the text box being cut off during the cropping process. After cropping, sample data is obtained, all sample data is aggregated to obtain a sample data set, and further, the field recognition system trains the initial field recognition model using the sample data set to obtain a field recognition application model. It should be noted that the field recognition application model is a fully convolutional neural network, which is a common image semantic segmentation model. Conventional training methods, such as the gradient-based backpropagation algorithm, can be used to update model parameters to improve recognition accuracy. The fully convolutional neural network is constructed through layering convolution and transposed convolution layers, and rich feature representation and accurate segmentation results are obtained through multi-level information extraction and upsampling operations. Through appropriate training and optimization of the network, accurate differentiation and positioning of different targets in the image can be achieved. In addition, some optimization strategies, such as data augmentation and regularization, can be used in the training process to improve the generalization ability and robustness of the model.

[0056] 103. In response to the field recognition instruction, the field recognition application model is input to the field recognition application model to obtain the target field position and the target field category corresponding to each target text field present in the to-be-recognized image.

[0057] The field identification instruction can be uploaded by a user based on a terminal device in which the user is located. The terminal device can be a smart device such as a mobile phone, a tablet computer, or a computer. The terminal device is not specifically limited in the present application.

[0058] Specifically, the image data to be identified can be obtained from the field identification instruction through a corresponding interface or tool. The image to be identified is input into the field identification application model, and a pre-trained full convolutional neural network model is used to perform a forward propagation operation on the input image to obtain the position and category of each target text field. During actual operation, the extracted target field position and category information can be displayed, recorded, or processed subsequently according to specific requirements.

[0059] The field identification method provided in this embodiment first obtains an original image set. Each original image in the original image set has multiple text fields. The field position and field category of each text field in each original image are labeled to obtain a labeled image set. Further, each labeled image in the labeled image set is cropped according to a preset cropping standard to obtain a sample data set. Next, the sample data set is used to train an initial field identification model of a full convolutional neural network to obtain a field identification application model. Finally, in response to a field identification instruction, an image to be identified carried in the field identification instruction is obtained, and the image to be identified is input into the field identification application model for identification to obtain the target field position and target field category corresponding to each target text field existing in the image to be identified. In the present embodiment, the full convolutional neural network can extract the features of an image through multiple layers of convolution and pooling operations. This allows the network to simultaneously meet the needs of large and small texts, allowing the network to accommodate text fields of different sizes and improving the accuracy of field identification. In addition, the full convolutional neural network can automatically learn the feature representation of the field position and category through supervised training. Compared with traditional methods, the model can automatically learn more expressive feature representations from a large amount of sample data without manually extracting features. This end-to-end learning method has better generalization ability and can adapt to different scenarios and text style recognition requirements.

[0060] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully describe the implementation process of the present embodiment, the present embodiment provides a field identification method as shown in Figure 2A The method comprises the following steps.

[0061] 201. An original image set is obtained, and each original image in the original image set is data-labeled to obtain a labeled image set.

[0062] In the embodiments of the present application, before the field recognition is performed, the relevant staff pre-collects original images with multiple text fields through various online and offline channels and uploads them to the field recognition system, so that the field recognition system trains the field recognition model according to the original images. The field recognition system stores the received original images into an original image set, and labels each original image using the image labeling tool preset by the relevant staff to obtain a labeled image set. The specific process of obtaining the labeled image set is as follows:

[0063] Firstly, the original images with multiple text fields can be invoices and other proof documents. The relevant staff can search for invoice images on the Internet to realize online collection, or obtain invoices from customers and partners to realize offline collection, so as to ensure that the original image set contains invoice images of different types, layouts and formats. After the relevant staff uploads the original images to the field recognition system, the field recognition system stores the original images into the original image set. Further, each original image is pre-processed and the labeled field categories are determined, such as invoice number, date, supplier name, amount, etc., and accurate labeling rules for each field category are defined. The image preprocessing includes but is not limited to noise removal processing, size unification processing and contrast enhancement processing. Next, the field recognition system uses the labeling tool preset by the relevant staff to draw a bounding box for each text field in each original image to label the field position of each text field, and sets a category label for each bounding box according to the labeling rules, associates the category label of each text field with the corresponding field position, and obtains a labeled image. Common data labeling tools include LabelImg, RectLabel, VGGImageAnnotator (VIA) and the like. For example,

[0064] 748,357,843,360,842,445,746,442invoice_date、

[0065] 352,211,585,211,585,292,352,292invoice_code、

[0066] 683,998,1458,998,1458,1067,683,1067invoice_number、

[0067] 361,366,426,366,426,431,361,431total_amount。

[0068] After completing data labeling, the field recognition system can directly aggregate the labeled images to obtain a labeled image set, or send the labeled images to a verification terminal for verification to obtain verification results, and store the labeled images indicated as passed by the verification results in the labeled image set. For the to-be-verified images indicated as failed by the verification results, it means that there is a labeling error. At this time, the revision image uploaded by the verification terminal based on the labeled image is stored as a labeled image in the labeled image set. Finally, the original image set is traversed, and each original image in the original image set is labeled to obtain a labeled image set.

[0069] Through the image labeling tool preset by the relevant staff, combined with the function of the field recognition system, a bounding box can be automatically drawn for each text field existing in each original image and the field category can be labeled, so as to realize automatic labeling of data and avoid time-consuming and labor cost of traditional manual labeling. At the same time, by introducing the verification terminal into the labeling process, the labeling result can be verified to ensure the accuracy and consistency of the labeling. The labeled images indicated as failed by the verification results can be further corrected and improved to improve the labeling quality.

[0070] 202. Crop each labeled image in the labeled image set according to the preset cropping standard to obtain a sample data set.

[0071] In the embodiments of the present application, for each labeled image in the labeled image set, the field recognition system needs to crop according to the field region labeled in the labeled image. Specifically, for any labeled image, the text box of each text field in the labeled image is determined, the labeled image is cropped according to the preset cropping standard, and after the cropping is completed, sample data is obtained, all sample data is aggregated, and a sample data set is obtained.

[0072] Specifically, the field recognition system first reads a plurality of cropping sizes corresponding to preset cropping standards, for example, the preset cropping sizes can be 100x100 pixels, 200x200 pixels, and 300x300 pixels, etc. For each labeled image (i.e., invoice image) in the labeled image set, the field positions of a plurality of text fields existing in the labeled image are queried, for example, invoice number, date, supplier name, and amount, etc. According to the field positions, a plurality of text boxes corresponding to the plurality of text fields are determined, for example, each invoice image can contain an invoice number box, a date box, a supplier name box, and an amount box. Next, the text boxes of each text field are determined in the labeled image, and any preset cropping size is selected for cropping according to the cropping condition, wherein the cropping condition is to ensure that the content of the text box is not cut off and the integrity of the text box is maintained. Subsequently, the plurality of cropped boxes obtained by cropping are scaled to a plurality of background boxes with consistent size, and the plurality of text boxes in the invoice image and the corresponding background boxes are used as sample data, and they are associated with each invoice image to obtain sample data corresponding to each invoice image, so as to facilitate subsequent model training to recognize the text fields. In addition, after determining the sample data corresponding to each invoice image and obtaining the sample data set, each sample data in the sample data set needs to be numbered, so as to extract the specified sample data corresponding to the specified number according to the preset split value, to obtain a plurality of specified sample data. For example, 100 sample data are extracted from the sample data set as training samples, and the remaining sample data are used as test samples. The plurality of specified sample data are aggregated into a training sample set for model training and parameter optimization. The remaining sample data are aggregated into a test sample set for model evaluation and performance testing.

[0073] 203、Using the sample data set to train the initial field recognition model to obtain a field recognition application model.

[0074] In the embodiment of the present application, the field recognition system adopts a sample data set to train an initial field recognition model to obtain a field recognition application model. It should be noted that the field recognition application model is a full convolutional neural network, which is a common image semantic segmentation model. A conventional training method, such as a gradient-based back propagation algorithm, can be used to update the model parameters to improve the recognition accuracy. The full convolutional neural network is constructed by stacking convolution and transpose convolution layers, and rich feature representations and accurate segmentation results are obtained through multi-level information extraction and upsampling operations. Through appropriate training and optimization of the network, accurate differentiation and positioning of different targets in the image can be achieved. The field recognition application model includes an input layer, a convolution layer, a classification layer, and an output layer. The classification layer includes a classification tensor output layer and a segmentation tensor output layer. The number of channels of the classification tensor output layer is equal to the number of classes of the field categories, and is used to determine the field category to which the pixel value belongs. The number of channels of the segmentation tensor output layer is 1, and is used to determine the coordinates of the text field.

[0075] In actual operation, the field recognition system inputs the training sample set in the sample data set to the initial field recognition model for field position recognition and field type recognition, updates the model parameters of the initial field recognition model according to the recognition result, and obtains a to-be-tested field recognition model. Further, the test sample set in the sample data set is input to the to-be-tested field recognition model for model testing, and a test result is obtained. When the accuracy of the test result meets a preset accuracy threshold, a field recognition application model is obtained. When the accuracy of the test result does not meet the preset accuracy threshold, the training sample is input to the initial field recognition model again for field position recognition and field type recognition, the model parameters of the initial field recognition model are updated according to the recognition result, and a to-be-tested field recognition model is obtained, until the test accuracy of the to-be-tested field recognition model meets the preset accuracy threshold, and a field recognition application model is obtained.

[0076] 204, in response to the field recognition instruction, obtaining the to-be-recognized image carried in the field recognition instruction.

[0077] In the embodiment of the present application, the field recognition instruction can be uploaded by a user based on the terminal device on which the user is located. The terminal device can be a mobile phone, a tablet computer, or a computer, and the like. The present application does not specifically limit the terminal device. In response to the field recognition instruction, the field recognition system can obtain the image data to be recognized from the field recognition instruction through a corresponding interface or tool, and then input the to-be-recognized image to the field recognition application model for recognition.

[0078] 205, inputting the to-be-recognized image to the field recognition application model for recognition to obtain the target field position and the target field category corresponding to each target text field existing in the to-be-recognized image.

[0079] In this embodiment, a pre-trained fully convolutional neural network model is used to perform a forward pass operation on the input image to obtain the position and category of each target text field. In actual operation, the extracted target field positions and categories can be displayed, recorded, or stored for subsequent processing according to specific needs.

[0080] Specifically, such as Figure 2B As shown, the convolutional layer of the field recognition application model extracts features from the image to be recognized, obtaining the target feature map. Then, in the classification layer of the field recognition application model, image processing is performed on the target feature map through segmentation and classification channels to obtain segmentation tensors and classification tensors. It should be noted that the convolutional layer is an FPN (Feature Pyramid Network) structure. FPN is a network structure commonly used in object detection and segmentation tasks. It achieves multi-scale target detection and accurate localization by fusing feature maps of different scales.

[0081] Furthermore, the output layer of the field recognition application model reads the pixel value of each pixel in the segmentation tensor, marking pixels with values ​​greater than a preset threshold as foreground regions and other pixels as background regions. Next, based on image processing algorithms, continuous boundaries in the foreground regions are calculated to obtain multiple text boxes for multiple target text fields. The position of each text box is used as the target field position of the target text field. For each text box, all corresponding text pixels are determined, and the field category of each text pixel is queried in the classification tensor. The field category with the highest pixel value is used as the target field category of the target text field corresponding to the text box. Other pixels are used to indicate pixels in the segmentation tensor other than the specified pixels. The fully convolutional neural network, through supervised training, can automatically learn feature representations of field positions and categories. Compared to traditional methods, manual feature extraction is no longer required; the model can autonomously learn more expressive feature representations based on a large amount of sample data. This end-to-end learning approach has better generalization ability and can adapt to the recognition needs of different scenarios and text styles.

[0082] The method provided in the embodiments of the present application firstly acquires an original image set, each original image in the original image set has multiple text fields, labels the field position and field category of each text field in each original image to obtain a labeled image set. Further, each labeled image in the labeled image set is cropped according to a preset cropping standard to obtain a sample data set. Next, the initial field recognition model of the full convolutional neural network is trained by using the sample data set to obtain a field recognition application model. Finally, in response to a field recognition instruction, a to-be-recognized image carried in the field recognition instruction is acquired and input into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the to-be-recognized image. In the embodiments of the present application, the full convolutional neural network can extract the features of the image through multiple convolution and pooling operations. This makes the network be able to simultaneously consider the needs of large texts and small texts, makes the network be able to consider text fields of different sizes, and improves the accuracy of field recognition. In addition, the full convolutional neural network can automatically learn the feature representation of the field position and category through supervised training. Compared with the traditional method, the feature representation with better expression ability can be learned by the model according to a large amount of sample data. This end-to-end learning method has better generalization ability and can adapt to the recognition needs of different scenes and text styles.

[0083] Further, as Figure 1 The specific implementation of the method, the embodiments of the present application provide a field recognition device, as shown in Figure 3 The device includes an acquisition module 301, a cropping module 302 and an identification module 303.

[0084] The acquisition module 301 is configured to acquire an original image set, perform data labeling on each original image in the original image set to obtain a labeled image set, wherein each original image has multiple text fields, and the data labeling is configured to label the field position and field category of each text field;

[0085] The cropping module 302 is configured to crop each labeled image in the labeled image set according to a preset cropping standard to obtain a sample data set, and train an initial field recognition model by using the sample data set to obtain a field recognition application model, wherein the field recognition application model is a full convolutional neural network.

[0086] The identification module 303 is configured to, in response to a field recognition instruction, acquire a to-be-recognized image carried in the field recognition instruction, and input the to-be-recognized image into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the to-be-recognized image.

[0087] In a specific application scenario, the identification module 303 is configured to perform feature extraction on the to-be-identified image by using a convolutional layer of the field identification application model to obtain a target feature map, and perform image processing on the target feature map by using a classification layer of the field identification application model to obtain a segmentation tensor and a classification tensor, where the convolutional layer is a feature pyramid network structure; the output layer of the field identification application model reads pixel values of each pixel in the segmentation tensor, marks a specified pixel with a pixel value greater than a preset pixel threshold as a foreground region, marks other pixels as a background region, and calculates a continuous boundary in the foreground region based on an image processing algorithm to obtain a plurality of text boxes of a plurality of target text fields, takes a position of a text box of each target text field as a target field position of the target text field, and for each text box in the plurality of text boxes, determines all text pixels corresponding to the text box, queries a field category of each text pixel in the classification tensor, and takes a field category with the most pixel values as a target field category of a target text field corresponding to the text box, and the other pixels are used to indicate pixels other than the specified pixels in the segmentation tensor.

[0088] In a specific application scenario, the classification layer includes a classification tensor output layer and a segmentation tensor output layer, where a number of channels of the classification tensor output layer is equal to a number of categories of field categories, and the classification tensor output layer is configured to determine a field category to which a pixel value belongs, and a number of channels of the segmentation tensor output layer is 1, and the segmentation tensor output layer is configured to determine a coordinate of a text field.

[0089] In a specific application scenario, the obtaining module 301 is configured to perform image preprocessing on each of the original images, where the image preprocessing includes but is not limited to noise removal processing, size unification processing, and contrast enhancement processing; use a preset labeling tool to label field positions of a plurality of text fields existing in each of the original images, set a category label for each text field, associate the category label of each text field with a corresponding field position, and obtain a labeled image; and label each of the original images in the original image set to obtain a labeled image set.

[0090] In a specific application scenario, the cropping module 302 is configured to read a plurality of cropping sizes corresponding to the preset cropping standard; for each of the labeled images in the labeled image set, query field positions of a plurality of text fields present in the labeled image, and determine a plurality of text boxes corresponding to the plurality of text fields according to the field positions; determine a text box of each of the text fields in the labeled image, continuously select any of the plurality of cropping sizes according to a cropping condition to crop, and scale a plurality of cropped boxes obtained by cropping to a plurality of background boxes with consistent size, the cropping condition being not cutting off the text box; take the plurality of text boxes and the plurality of background boxes corresponding to the labeled image as sample data corresponding to the labeled image; determine sample data corresponding to each of the labeled images to obtain the sample data set.

[0091] In a specific application scenario, the cropping module 302 is further configured to number each of the sample data in the sample data set, extract specified sample data corresponding to a specified number to obtain a plurality of specified sample data, the specified number being any value, and the number of the plurality of specified sample data satisfying a preset splitting value; aggregate the plurality of specified sample data into a training sample set, and aggregate the remaining sample data after extraction into a test sample set.

[0092] In a specific application scenario, the cropping module 302 is configured to input the training sample set in the sample data set to the initial field recognition model for field position recognition and field type recognition, update model parameters of the initial field recognition model according to a recognition result to obtain a to-be-tested field recognition model; input the test sample set in the sample data set to the to-be-tested field recognition model for model testing to obtain a test result; and when an accuracy of the test result satisfies a preset accuracy threshold, obtain the field recognition application model.

[0093] The device provided in the embodiments of the present application first acquires an original image set, each original image in the original image set has multiple text fields, and the field position and field category of each text field in each original image are labeled to obtain a labeled image set. Further, each labeled image in the labeled image set is cropped according to a preset cropping standard to obtain a sample data set. Next, the initial field recognition model of the full convolutional neural network is trained by using the sample data set to obtain a field recognition application model. Finally, in response to a field recognition instruction, a to-be-recognized image carried in the field recognition instruction is acquired, and the to-be-recognized image is input into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the to-be-recognized image. In the embodiments of the present application, the full convolutional neural network can extract the features of the image through multiple convolution and pooling operations. This enables the network to simultaneously meet the needs of large texts and small texts, enables the network to meet different sizes of text fields, and improves the accuracy of field recognition. In addition, the full convolutional neural network can automatically learn the feature representation of the field position and category through supervised training. Compared with the traditional method, the feature representation with better expression ability can be learned by the model according to a large amount of sample data. This end-to-end learning method has better generalization ability and can adapt to the recognition needs of different scenes and text styles.

[0094] It should be noted that other corresponding descriptions of the functions of the field recognition device provided in the embodiments of the present application can be referred to the corresponding descriptions in Figure 1 and Figure 2A to Figure 2B , which will not be described here.

[0095] Based on the above method as shown in Figure 1 , Figure 2A to Figure 2B , correspondingly, the present embodiment also provides a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the field recognition method.

[0096] Based on such understanding, the technical solution of the present application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment scenario of the present application.

[0097] Based on the above method as shown in Figure 1 , Figure 2A to Figure 2B , and Figure 3 the field recognition device embodiment, in order to achieve the above purpose, in the exemplary embodiments, see Figure 4The device also includes a communication bus, a processor, a memory, and a communication interface, and can further include an input / output interface and a display device, wherein the respective functional units can communicate with each other through the bus. The memory stores a computer program, and the processor is configured to execute the program stored in the memory to perform the field recognition method in the above embodiments.

[0098] Optionally, the entity device can further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, and the like. The user interface can include a display screen, an input unit such as a keyboard, and the like. Optionally, the user interface can further include a USB interface, a card reader interface, and the like. The network interface can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the like.

[0099] The storage medium can further include an operating system and a network communication module. The operating system is a program for managing the hardware of the entity device and the to-be-identified software resources, and supports the running of the information processing program and other to-be-identified software and / or programs. The network communication module is used to realize the communication between the components in the storage medium, and the communication with other hardware and software in the information processing entity device.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software with a necessary general hardware platform, or by hardware. By applying the technical solutions of the present application, first, an original image set is acquired, each original image in the original image set has multiple text fields, the field position and the field category of each text field in each original image are labeled to obtain a labeled image set. Further, according to a preset cropping standard, each labeled image in the labeled image set is cropped to obtain a sample data set. Next, the initial field recognition model of the full convolutional neural network is trained by using the sample data set to obtain a field recognition application model. Finally, in response to a field recognition instruction, a to-be-recognized image carried in the field recognition instruction is acquired, and the to-be-recognized image is input into the field recognition application model for recognition to obtain the target field position and the target field category corresponding to each target text field existing in the to-be-recognized image. Compared with the prior art, in the embodiment of the present application, the full convolutional neural network can extract the features of the image through multiple layers of convolution and pooling operations. This enables the network to simultaneously meet the needs of large and small texts, enabling the network to accommodate text fields of different sizes and improving the accuracy of field recognition. In addition, the full convolutional neural network can automatically learn the feature representation of the field position and the category through supervised training. Compared with traditional methods, manual feature extraction is no longer needed, and the model can automatically learn more expressive feature representations from a large amount of sample data. This end-to-end learning method has better generalization ability and can adapt to different scenarios and text style recognition requirements.

[0101] Those skilled in the art can understand that the drawings are only schematic of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily required for implementing the present application.

[0102] Those skilled in the art can understand that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the description of the implementation scenario, or can be changed and located in one or more devices different from the implementation scenario. The modules of the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0103] The above application number is only for description, not representing the advantages and disadvantages of the implementation scenario.

[0104] The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims

1. A field recognition method characterized by, The method comprises the following steps: obtaining an original image set, performing data labeling on each original image in the original image set to obtain a labeled image set, wherein each original image contains a plurality of text fields, and the data labeling is used to label the field position and field category of each text field; according to a preset cropping standard, cropping each labeled image in the labeled image set to obtain a sample data set, and using the sample data set to train an initial field recognition model to obtain a field recognition application model, wherein the field recognition application model is a full convolutional neural network; in response to a field recognition instruction, obtaining a to-be-recognized image carried in the field recognition instruction, and inputting the to-be-recognized image into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the to-be-recognized image; the step of inputting the to-be-recognized image into the field recognition application model for recognition to obtain the target field position and target field category corresponding to each target text field existing in the to-be-recognized image comprises the following steps: using a convolutional layer of the field recognition application model to extract features of the to-be-recognized image to obtain a target feature map, and using a classification layer of the field recognition application model to process the target feature map to obtain a segmentation tensor and a classification tensor, wherein the convolutional layer is a feature pyramid network structure; reading the pixel value of each pixel in the segmentation tensor at an output layer of the field recognition application model, marking a specified pixel with a pixel value greater than a preset pixel threshold as a foreground region, marking other pixels as a background region, and calculating the continuous boundary in the foreground region based on an image processing algorithm to obtain a plurality of text boxes of a plurality of target text fields, taking the position of the text box of each target text field as the target field position of the target text field, and for each text box in the plurality of text boxes, determining all text pixels corresponding to the text box, querying the field category of each text pixel in the classification tensor, and taking the field category with the most pixel value as the target field category of the target text field corresponding to the text box, wherein the other pixels are used to indicate the pixels in the segmentation tensor other than the specified pixels; the classification layer comprises a classification tensor output layer and a segmentation tensor output layer, wherein the number of channels of the classification tensor output layer is equal to the number of categories of the field categories, and is used to determine the field category to which the pixel value belongs, and the number of channels of the segmentation tensor output layer is 1, and is used to determine the coordinates of the text field.

2. The method of claim 1, wherein, the step of obtaining an original image set, performing data labeling on each original image in the original image set to obtain a labeled image set comprises the following steps: performing image preprocessing on each original image, wherein the image preprocessing includes but is not limited to noise removal processing, size unification processing and contrast enhancement processing; using a preset labeling tool to label the field position of each text field existing in each original image, setting a category label for each text field, associating the category label of each text field with the corresponding field position, and obtaining a labeled image. annotating each of the original images in the original image set to obtain a labeled image set.

3. The method of claim 1, wherein, The method further includes: reading a plurality of cropping sizes corresponding to the preset cropping standard; For each of the labeled images in the labeled image set, querying the field positions of the plurality of text fields present in the labeled image, and determining a plurality of text boxes corresponding to the plurality of text fields according to the field positions; In the labeled image, determine the text box of each of the text fields, and continuously select any of the plurality of cropping sizes for cropping according to the cropping condition, and scale the plurality of cropping boxes obtained by cropping to a plurality of background boxes with consistent size, the cropping condition being not cutting off the text box; The plurality of text boxes and the plurality of background boxes corresponding to the labeled image are used as sample data corresponding to the labeled image; determining the sample data corresponding to each of the labeled images to obtain the sample data set.

4. The method of claim 3, wherein, After the sample data set is obtained by determining the sample data corresponding to each of the labeled images, the method further includes: numbering each sample data in the sample data set, extracting a specified sample data corresponding to a specified number to obtain a plurality of specified sample data, the specified number being any value, and the number of the plurality of specified sample data satisfying a preset splitting value; aggregate the plurality of specified sample data into a training sample set, and aggregate the remaining sample data after extraction into a test sample set.

5. The method of claim 1, wherein, The method further includes: inputting the training sample set in the sample data set into the initial field recognition model for field position recognition and field type recognition, updating the model parameters of the initial field recognition model according to the recognition result to obtain a to-be-tested field recognition model; inputting the test sample set in the sample data set into the to-be-tested field recognition model for model testing to obtain a test result; when the accuracy of the test result meets a preset accuracy threshold, the field recognition application model is obtained.

6. A field identification apparatus, characterized by comprising: The method further includes: an acquisition module configured to acquire an original image set, and perform data annotation on each of the original images in the original image set to obtain a labeled image set, wherein each of the original images has a plurality of text fields, and the data annotation is configured to annotate field positions and field categories of each of the text fields; a cropping module configured to crop each of the labeled images in the labeled image set according to a preset cropping standard to obtain a sample data set, and train an initial field recognition model using the sample data set to obtain a field recognition application model, the field recognition application model being a full convolutional neural network; The identification module is configured to, in response to a field identification instruction, acquire a to-be-identified image carried in the field identification instruction, input the to-be-identified image into the field identification application model for identification, and obtain a target field position and a target field category corresponding to each target text field present in the to-be-identified image. The inputting of the to-be-identified image into the field identification application model for identification and the obtaining of the target field position and the target field category corresponding to each target text field present in the to-be-identified image include: performing feature extraction on the to-be-identified image by using a convolution layer of the field identification application model to obtain a target feature map, and performing image processing on the target feature map by using a classification layer of the field identification application model to obtain a segmentation tensor and a classification tensor, the convolution layer being a feature pyramid network structure; reading, by using an output layer of the field identification application model, a pixel value of each pixel in the segmentation tensor, marking a specified pixel with a pixel value greater than a preset pixel threshold as a foreground region, marking other pixels as a background region, and calculating a continuous boundary in the foreground region based on an image processing algorithm to obtain a plurality of text boxes of a plurality of target text fields, taking a position of a text box of each target text field as a target field position of the target text field, and for each text box in the plurality of text boxes, determining all text pixels corresponding to the text box, querying a field category of each text pixel in the classification tensor, and taking a field category with the most pixel values as a target field category of a target text field corresponding to the text box, the other pixels being used to indicate pixels other than the specified pixels in the segmentation tensor; the classification layer includes a classification tensor output layer and a segmentation tensor output layer, wherein a number of channels of the classification tensor output layer is equal to a number of categories of field categories, and the classification tensor output layer is used to determine a field category to which a pixel value belongs, a number of channels of the segmentation tensor output layer is 1, and the segmentation tensor output layer is used to determine a coordinate of a text field. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 5.

Images