Express sheet text detection method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to an express sheet text detection method and device, equipment and a storage medium, and the method comprises the steps: obtaining historical express sheet image data, and preprocessing the historical express sheet image data to obtain historical preprocessed image data; performing data enhancement processing and division processing on the historical preprocessed image data to obtain a training set, a verification set and a test set; constructing a multi-task learning framework and a multi-task learning model; respectively inputting the training set and the verification set into the constructed multi-task learning model for training, and testing the trained multi-task learning model by adopting the test set to obtain a text detection model; acquiring real-time express sheet image data and preprocessing the real-time express sheet image data to obtain real-time preprocessed image data; inputting the real-time preprocessed image data into the text detection model to obtain an express sheet text detection result; the method disclosed by the invention can solve the problem of irregular surface text detection, and has the advantages of high detection precision and high efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for detecting text on a webpage. Background Art

[0002] With the rapid development of the express delivery industry, the design of express delivery bills has become increasingly diversified. Specifically, there are express delivery bills of various shapes and styles. This diversity has brought new challenges in the processing of express delivery bills. In addition, in logistics transportation and daily operations, express delivery bill images are often distorted and deformed, resulting in irregular bends in text images. This distortion and deformation poses a severe test to traditional text detection technology.

[0003] The existing technology performs well in detecting express delivery bills of standard shapes, but when faced with express delivery bills of single format and arbitrary shapes, problems such as inaccurate detection, missed detection, and false detection often occur. These problems seriously affect the efficiency of the express delivery industry and make it difficult to meet the demand for efficient and automated processing.

[0004] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for detecting text on a face sheet, which can solve the problem of detecting text on irregular face sheets and has the advantages of high detection accuracy and high detection efficiency.

[0006] The first aspect of the present invention provides a method for detecting text on a face bill, comprising: obtaining historical face bill image data, and preprocessing the historical face bill image data to obtain historical preprocessed image data; performing data enhancement processing and partitioning processing on the historical preprocessed image data to obtain a training set, a validation set, and a test set; constructing a multi-task learning framework, and constructing a multi-task learning model based on the constructed multi-task learning framework; respectively inputting the training set and the validation set into the constructed multi-task learning model for training, and using the test set to test the trained multi-task learning model to obtain a text detection model; obtaining real-time face bill image data, and preprocessing the real-time face bill image data to obtain real-time preprocessed image data; inputting the real-time preprocessed image data into the text detection model to obtain a face bill text detection result.

[0007] Optionally, in a first implementation method of the first aspect of the present invention, the acquiring of historical mailing list image data and preprocessing of the historical mailing list image data to obtain historical preprocessed image data includes: acquiring historical mailing list image data, converting multiple historical mailing list images included in the historical mailing list image data into the same format to obtain multiple standardized historical mailing list images; performing denoising, binarization and image correction on the standardized historical mailing list images one by one to obtain multiple corrected historical mailing list images; using image processing technology to locate and mark the text areas in the corrected historical mailing list images one by one to obtain multiple preprocessed historical images; and integrating multiple preprocessed historical images to obtain historical preprocessed image data.

[0008] Optionally, in a second implementation method of the first aspect of the present invention, the historical preprocessed image data are respectively subjected to data enhancement processing and division processing to obtain a training set, a verification set and a test set, including: using an image processing library to perform data enhancement processing on the historical preprocessed image data based on data enhancement technology to obtain enhanced historical image data; removing enhanced historical images with abnormal markings from the enhanced historical image data to obtain historical image data to be divided; using a random sampling method to divide the historical image data to be divided into a training set, a verification set and a test set based on a preset division ratio.

[0009] Optionally, in a third implementation method of the first aspect of the present invention, the multi-task learning framework is constructed, and a multi-task learning model is constructed based on the constructed multi-task learning framework, including: constructing a multi-task learning framework that combines regression tasks and segmentation tasks, and using CSPDarknet53 as the backbone of the multi-task learning framework; setting the head network of the regression task and the head network of the segmentation task respectively; defining a mean square error function as the loss function of the regression task, defining a cross entropy loss function as the loss function of the segmentation task, and defining a comprehensive loss function of the regression task and the segmentation task to complete the construction of the task learning model.

[0010] Optionally, in a fourth implementation method of the first aspect of the present invention, the head network for the regression task and the head network for the segmentation task are set separately, including: designing a fully connected layer for implementing the regression task, and adding a linear activation function after the fully connected layer to complete the setting of the head network for the regression task; designing a convolutional neural network structure for implementing the segmentation task, the convolutional neural network structure including multiple convolutional layers, activation layers and pooling layers; at the output end of the pooling layer, embedding an upsampling technology for restoring the spatial resolution of the image; after the pooling layer, adding a 1x1 convolutional layer for pixel-level classification to perform pixel-level classification to complete the setting of the head network for the segmentation task.

[0011] Optionally, in a fifth implementation method of the first aspect of the present invention, the training set and the validation set are respectively input into the constructed multi-task learning model for training, and the trained multi-task learning model is tested using the test set to obtain a text detection model, including: inputting the training set into the constructed multi-task learning model for training, and using the validation set to perform performance evaluation on the multi-task learning model of the training process; based on the performance evaluation result of the validation set, the hyperparameters of the multi-task learning model are adjusted in combination with the early stopping method and the Bayesian optimization method to obtain the model to be tested; the test set is input into the model to be tested, and based on the test result, the model to be tested is optimized in combination with the regularization technique and the Bayesian optimization method to obtain the text detection model.

[0012] Optionally, in a sixth implementation method of the first aspect of the present invention, the real-time waybill image data is acquired and pre-processed to obtain real-time pre-processed image data, including: acquiring real-time waybill image data, performing format conversion processing on the real-time waybill data to obtain standardized real-time waybill data; performing denoising processing, binarization processing and image correction processing on the standardized real-time waybill data to obtain corrected real-time waybill data; using image processing technology to perform text area positioning processing on the corrected real-time waybill data, and performing positioning marking to obtain real-time pre-processed image data.

[0013] The second aspect of the present invention provides a method for detecting text on a face bill, comprising: a first processing module, used to obtain historical face bill image data, and preprocess the historical face bill image data to obtain historical preprocessed image data; an enhancement and division module, used to perform data enhancement processing and division processing on the historical preprocessed image data, respectively, to obtain a training set, a verification set, and a test set; a construction module, used to construct a multi-task learning framework, and construct a multi-task learning model based on the constructed multi-task learning framework; a training and testing module, used to input the training set and the verification set into the constructed multi-task learning model for training, and use the test set to test the trained multi-task learning model to obtain a text detection model; a second processing module, used to obtain real-time face bill image data, and preprocess the real-time face bill image data to obtain real-time preprocessed image data; a detection module, used to input the real-time preprocessed image data into the text detection model to obtain a face bill text detection result.

[0014] Optionally, in a first implementation method of the second aspect of the present invention, the first processing module includes: a first conversion unit, used to acquire historical mailing list image data, convert multiple historical mailing list images included in the historical mailing list image data into the same format, and obtain multiple standardized historical mailing list images; a first processing unit, used to perform denoising, binarization and image correction on the standardized historical mailing list images one by one, and obtain multiple corrected historical mailing list images; a first positioning unit, used to use image processing technology to locate and mark the text areas in the corrected historical mailing list images one by one, and obtain multiple pre-processed historical images; an integration unit, used to integrate multiple pre-processed historical images to obtain historical pre-processed image data.

[0015] Optionally, in a second implementation of the second aspect of the present invention, the enhanced partitioning module includes: an enhancement unit, used to use an image processing library to perform data enhancement processing on historical preprocessed image data based on data enhancement technology to obtain enhanced historical image data; a removal unit, used to remove enhanced historical images with abnormal markings from the enhanced historical image data to obtain historical image data to be divided; and a partitioning unit, used to use a random sampling method to divide the historical image data to be divided into a training set, a verification set, and a test set based on a preset partitioning ratio.

[0016] Optionally, in a third implementation method of the second aspect of the present invention, the construction module includes: a construction unit, used to construct a multi-task learning framework combining regression tasks and segmentation tasks, and using CSPDarknet53 as the backbone of the multi-task learning framework; a setting unit, used to set the head network of the regression task and the head network of the segmentation task respectively; a definition unit, used to define a mean square error function as the loss function of the regression task, define a cross entropy loss function as the loss function of the segmentation task, and define a comprehensive loss function of the regression task and the segmentation task to complete the construction of the task learning model.

[0017] Optionally, in a fourth implementation method of the second aspect of the present invention, the building module also includes: a first design unit, used to design a fully connected layer for implementing a regression task, and adding a linear activation function after the fully connected layer to complete the setting of the head network for the regression task; a second design unit, used to design a convolutional neural network structure for implementing a segmentation task, the convolutional neural network structure including multiple convolutional layers, activation layers and pooling layers; an embedding unit, used to embed an upsampling technology for restoring the spatial resolution of the image at the output end of the pooling layer; an additional unit, used to add a 1x1 convolutional layer for pixel-level classification after the pooling layer to perform pixel-level classification to complete the setting of the head network for the segmentation task.

[0018] Optionally, in a fifth implementation of the second aspect of the present invention, the training and testing module includes: a training unit, used to input the training set into the constructed multi-task learning model for training, and use a validation set to perform performance evaluation on the multi-task learning model of the training process; an adjustment unit, used to adjust the hyperparameters of the multi-task learning model based on the performance evaluation results of the validation set, combined with early stopping method and Bayesian optimization method, to obtain a model to be tested; a testing unit, used to input the test set into the model to be tested, and based on the test results, optimize the model to be tested in combination with regularization technology and Bayesian optimization method to obtain a text detection model.

[0019] Optionally, in a sixth implementation method of the second aspect of the present invention, the second processing module includes: a second conversion unit, used to acquire real-time waybill image data, perform format conversion processing on the real-time waybill data, and obtain standardized real-time waybill data; a second processing unit, used to perform denoising processing, binarization processing, and image correction processing on the standardized real-time waybill data, and obtain corrected real-time waybill data; a second positioning unit, used to use image processing technology to perform text area positioning processing on the corrected real-time waybill data, and perform positioning marking to obtain real-time pre-processed image data.

[0020] The third aspect of the present invention provides a face sheet text detection device, which comprises: a memory and at least one processor, wherein the memory stores instructions; at least one of the processors calls the instructions in the memory so that the face sheet text detection device executes each step of any one of the face sheet text detection methods described above.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and when the instructions are executed by a processor, the various steps of any of the above-mentioned methods for detecting text on a face sheet are implemented.

[0022] In the technical solution of the present invention, historical face sheet image data is obtained by acquiring and preprocessing historical face sheet image data; data enhancement processing and partitioning processing are respectively performed on the historical preprocessed image data to obtain a training set, a validation set and a test set; a multi-task learning framework and a multi-task learning model are constructed; the training set and the validation set are respectively input into the constructed multi-task learning model for training, and the trained multi-task learning model is tested with the test set to obtain a text detection model; real-time face sheet image data is acquired and preprocessed to obtain real-time preprocessed image data; the real-time preprocessed image data is input into the text detection model to obtain a face sheet text detection result; the method disclosed in the present application can solve the problem of irregular face sheet text detection, and has the advantages of high detection accuracy and high detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A first flow chart of a data processing method provided by an embodiment of the present invention;

[0024] Figure 2 A second flow chart of the data processing method provided by an embodiment of the present invention;

[0025] Figure 3 A third flow chart of the data processing method provided by an embodiment of the present invention;

[0026] Figure 4 A fourth flow chart of the data processing method provided by an embodiment of the present invention;

[0027] Figure 5 A fifth flow chart of the data processing method provided by the embodiment of the present invention;

[0028] Figure 6 A sixth flow chart of the data processing method provided by an embodiment of the present invention;

[0029] Figure 7 A seventh flow chart of the data processing method provided by the embodiment of the present invention;

[0030] Figure 8 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention;

[0031] Fig. 9 A schematic diagram of the structure of a data processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention provides a method, device, equipment and storage medium for detecting text on a sheet. In the present invention, the terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the method for detecting text on a sheet in an embodiment of the present invention includes:

[0034] 101. Acquire historical waybill image data, and preprocess the historical waybill image data to obtain historical preprocessed image data;

[0035] In this embodiment, by preprocessing the historical face order image data, noise and irrelevant information in the image can be removed, and the clarity and contrast of the image can be improved, thereby making subsequent image analysis and processing more accurate; and the preprocessing helps to highlight the key features in the image, so that the content in the image can be more accurately identified during subsequent processing.

[0036] 102. Perform data enhancement processing and partition processing on the historical preprocessed image data to obtain a training set, a validation set, and a test set;

[0037] In this embodiment, by performing data enhancement and division processing on the historical preprocessed image data, the diversity and generalization ability of the text detection model training can be increased, while ensuring the reasonable division of the training set, validation set and test set, which is helpful for the accurate evaluation of the multi-task learning model.

[0038] 103. Construct a multi-task learning framework, and construct a multi-task learning model based on the constructed multi-task learning framework;

[0039] In this embodiment, the constructed multi-task learning framework is able to process multiple related tasks simultaneously, which helps the model share feature representations during the learning process and improves learning efficiency and model performance.

[0040] 104. Input the training set and the validation set into the constructed multi-task learning model for training respectively, and use the test set to test the trained multi-task learning model to obtain a text detection model;

[0041] 105. Acquire real-time waybill image data, and pre-process the real-time waybill image data to obtain real-time pre-processed image data;

[0042] 106. Input the real-time pre-processed image data into the text detection model to obtain the face sheet text detection result.

[0043] In this embodiment, real-time waybill image data is acquired and preprocessed, and then input into a trained text detection model, which can achieve fast and accurate waybill text detection, reduce manual intervention processes, and improve business processing efficiency.

[0044] The present application discloses a method for detecting text on a face sheet, which comprises the following steps: obtaining historical face sheet image data and performing preprocessing to obtain historical preprocessed image data; performing data enhancement processing and partitioning processing on the historical preprocessed image data to obtain a training set, a validation set and a test set; constructing a multi-task learning framework and a multi-task learning model; inputting the training set and the validation set into the constructed multi-task learning model for training, and using the test set to test the trained multi-task learning model to obtain a text detection model; obtaining real-time face sheet image data and performing preprocessing to obtain real-time preprocessed image data; inputting the real-time preprocessed image data into the text detection model to obtain a face sheet text detection result; the method disclosed in the present application can solve the problem of irregular face sheet text detection, and has the advantages of high detection accuracy and high detection efficiency.

[0045] See also Figure 2 The second embodiment of the method for detecting text on a sheet in the embodiment of the present invention includes:

[0046] 201. Acquire historical mail order image data, convert multiple historical mail order images included in the historical mail order image data into the same format, and obtain multiple standardized historical mail order images;

[0047] 202. Perform denoising, binarization and image correction on the standardized historical face order images one by one to obtain a plurality of corrected historical face order images;

[0048] In this embodiment, the historical face order image is denoised by a median filtering method to reduce noise interference in the image and retain important details of the image; then, the image is converted into a binary image through a preset threshold, that is, each pixel in the image has only two possible values, and the pixel brightness higher than the preset threshold becomes white, and the pixel brightness lower than the preset threshold becomes black. Binarization helps to simplify the image and facilitate subsequent processing; finally, the binarized image is corrected to correct problems such as image tilt and distortion, and feature point matching technology is used to perform image rotation, scaling, perspective transformation and other operations on the historical face order image to ensure the alignment and standardization of the historical face order image.

[0049] 203. Using image processing technology, the text areas in the corrected historical image are located one by one, and marked, to obtain a plurality of pre-processed historical images;

[0050] In this embodiment, the historical mail order images are loaded through the OpenCV library, and then the Canny edge algorithm is applied to detect and identify the outlines of the text areas in the historical mail order images. Morphological operations such as dilation and erosion are then used to enhance the features of the text areas and remove noise. Finally, connected region analysis is used to locate the text areas, and contour detection is used to mark the text areas on these historical mail order images.

[0051] 204. Integrate multiple pre-processed historical images to obtain historical pre-processed image data.

[0052] See also Figure 3 The third embodiment of the method for detecting text on a sheet in the embodiment of the present invention includes:

[0053] 301. Using an image processing library, based on data enhancement technology, perform data enhancement processing on the historical pre-processed image data to obtain enhanced historical image data;

[0054] In this embodiment, the OpenCV library is used to perform data enhancement processing on the historical preprocessed image data in combination with data enhancement methods such as rotation, scaling, cropping, color conversion, adding noise, horizontal or vertical flipping, etc., to generate new image variants and obtain enhanced historical image data.

[0055] 302. Eliminate enhanced historical images with abnormal markings from the enhanced historical image data to obtain historical image data to be divided;

[0056] In this embodiment, an image quality assessment algorithm is used to remove enhanced historical images with abnormal markings to obtain historical image data to be divided.

[0057] 303. Use random sampling method to divide the historical image data into training set, verification set and test set based on the preset division ratio

[0058] In this embodiment, 80% of the images can be selected as training sets, 10% as validation sets, and 10% as test sets; all historical image data to be divided are collected into one data set, and the order of images in the data set is disrupted using a randomization algorithm to ensure that each image is randomly assigned to a different set; according to the set ratio, images are selected from the disrupted data set in turn and assigned to the training set, validation set, and test set. For example, if there are 1,000 images in total, the training set will contain the first 800 images, the next 100 images will be used as a validation set, and the last 100 images will be used as a test set; the randomization algorithm is used for division to ensure that when dividing the data set, the images in each set are distributed as evenly as possible to avoid data bias affecting the generalization ability of the model.

[0059] See also Figure 4 The fourth embodiment of the method for detecting text on a sheet in the embodiment of the present invention includes:

[0060] 401. Construct a multi-task learning framework that combines regression tasks and segmentation tasks, and use CSPDarknet53 as the backbone of the multi-task learning framework;

[0061] In this embodiment, by constructing a multi-task learning framework that combines regression tasks and segmentation tasks, the model can learn multiple related tasks at the same time, thereby reducing repeated calculations and training time, and helping the model to better generalize to unseen data; CSPDarknet53 is used as the backbone network. The network structure design helps to improve computing efficiency and feature extraction capabilities while reducing model complexity.

[0062] 402. Setting a head network for a regression task and a head network for a segmentation task respectively;

[0063] 403. Define the mean square error function as the loss function of the regression task, define the cross entropy loss function as the loss function of the segmentation task, and define the comprehensive loss function of the regression task and the segmentation task to complete the construction of the task learning model;

[0064] In this embodiment, specific network heads are optimized and trained separately, and by setting different loss functions for regression tasks and segmentation tasks, model learning can be guided more accurately; the use of the mean square error function in the regression task can effectively measure the difference between the predicted value and the true value, while the use of the cross entropy loss function in the segmentation task helps to improve the accuracy of classification; the definition of the comprehensive loss function enables the model to consider the performance of two tasks at the same time, thereby improving the generalization ability and accuracy of the learning model as a whole.

[0065] In this embodiment, the comprehensive function is in the form of:

[0066] L=α·L reg +β·L seg ;

[0067] Among them, L reg is the loss function of the regression error part, L seg is the loss function of the segmentation accuracy part, α and β are weight parameters used to balance the importance of regression error and segmentation accuracy.

[0068] See also Figure 5 The fifth embodiment of the method for detecting text on a sheet in the embodiment of the present invention includes:

[0069] 501. Design a fully connected layer for implementing the regression task, and add a linear activation function after the fully connected layer to complete the setting of the head network for the regression task;

[0070] In this embodiment, by designing a fully connected layer for implementing the regression task and adding a linear activation function after the fully connected layer, the accuracy of the continuous values ​​output by the model in the regression task can be ensured, which can adapt to scenarios requiring high-precision prediction.

[0071] 502. Design a convolutional neural network structure for implementing a segmentation task, wherein the convolutional neural network structure includes a plurality of convolutional layers, activation layers, and pooling layers;

[0072] In this embodiment, a convolutional neural network structure is designed to implement the segmentation task, including multiple convolutional layers, activation layers, and pooling layers, which can extract the hierarchical features of the image layer by layer and enhance the model's ability to understand and process complex image content.

[0073] 503. At the output of the pooling layer, an upsampling technique for restoring the spatial resolution of the image is embedded;

[0074] In this embodiment, upsampling technology is embedded at the output of the pooling layer to help restore the spatial resolution of the image, which is particularly important for image segmentation tasks because it requires accurate positioning and classification of each pixel in the image.

[0075] 504. After the pooling layer, a 1x1 convolutional layer is added for pixel-level classification to complete the setting of the head network for the segmentation task.

[0076] In this embodiment, a 1x1 convolution layer is added after the pooling layer for pixel-level classification, which enables the network to accurately classify each pixel, thereby improving the accuracy and detail expression of the segmentation task; that is, by combining the fully connected layer, convolutional neural network structure, upsampling technology and 1x1 convolution layer, an efficient and accurate network design is provided for regression and segmentation tasks.

[0077] See also Figure 6 The sixth embodiment of the method for detecting text on a sheet in the embodiment of the present invention includes:

[0078] 601. Input the training set into the constructed multi-task learning model for training, and use the validation set to evaluate the performance of the multi-task learning model in the training process;

[0079] 602. Based on the performance evaluation results of the validation set, the hyperparameters of the multi-task learning model are adjusted by combining the early stopping method and the Bayesian optimization method to obtain the model to be tested;

[0080] In this embodiment, performance evaluation based on the validation set can ensure the generalization ability of the model on unseen data and avoid overfitting; secondly, the early stopping method can prevent the model from overfitting the training data during the training process, and maintain the generalization ability of the model by stopping training when the performance of the validation set no longer improves; finally, the Bayesian optimization method is used for hyperparameter adjustment, which is an efficient global optimization strategy that can explore the hyperparameter space more intelligently and find a better hyperparameter combination, thereby improving model performance.

[0081] 603. Input the test set into the model to be tested, and optimize the model to be tested based on the test results by combining regularization technology and Bayesian optimization method to obtain a text detection model.

[0082] In this embodiment, by inputting the test set into the model to be tested and combining regularization technology and Bayesian optimization method, automatic optimization of the model can be achieved, manual intervention can be reduced, and efficiency can be improved; regularization technology helps to prevent model overfitting, while the Bayesian optimization method can effectively search for the optimal model parameters. The combination of the two can significantly improve the generalization ability and detection accuracy of the model; further, the Bayesian optimization method takes into account the information of previous evaluations during the parameter search process, and can more efficiently find the optimal parameter combination, which can save a lot of computing resources and time compared to traditional grid search or random search methods.

[0083] See also Figure 7 The seventh embodiment of the method for detecting text on a sheet in the embodiment of the present invention includes:

[0084] 701. Acquire real-time waybill image data, perform format conversion on the real-time waybill data, and obtain standardized real-time waybill data;

[0085] 702. Perform denoising, binarization and image correction on the standardized real-time waybill data to obtain corrected real-time waybill data;

[0086] 703. Using image processing technology, perform text area positioning processing on the corrected real-time face sheet data, and perform positioning marking to obtain real-time pre-processed image data;

[0087] In this embodiment, the format conversion processing method, denoising processing method, binarization processing method, image correction processing method and text area positioning method of historical waybill data can be referred to in order to pre-process the real-time waybill data.

[0088] The above describes the method for detecting text on a sheet in an embodiment of the present invention. The following describes the device for detecting text on a sheet in an embodiment of the present invention. Figure 8 In one embodiment of the present invention, a device for detecting text on a sheet includes:

[0089] The first processing module 801 is used to obtain historical face order image data and preprocess the historical face order image data to obtain historical preprocessed image data; the enhancement and partitioning module 802 is used to perform data enhancement processing and partitioning processing on the historical preprocessed image data to obtain a training set, a verification set and a test set; the construction module 803 is used to construct a multi-task learning framework and construct a multi-task learning model based on the constructed multi-task learning framework; the training and testing module 804 is used to input the training set and the verification set into the constructed multi-task learning model for training, and use the test set to test the trained multi-task learning model to obtain a text detection model; the second processing module 805 is used to obtain real-time face order image data and preprocess the real-time face order image data to obtain real-time preprocessed image data; the detection module 806 is used to input the real-time preprocessed image data into the text detection model to obtain the face order text detection result.

[0090] In this embodiment, the first processing module 801 includes: a first conversion unit 8011, which is used to obtain historical mailing list image data, convert multiple historical mailing list images included in the historical mailing list image data into the same format, and obtain multiple standardized historical mailing list images; a first processing unit 8012, which is used to perform denoising, binarization and image correction on the standardized historical mailing list images one by one, and obtain multiple corrected historical mailing list images; a first positioning unit 8013, which is used to use image processing technology to locate and mark the text areas in the corrected historical mailing list images one by one, and obtain multiple pre-processed historical images; an integration unit 8014, which is used to integrate multiple pre-processed historical images to obtain historical pre-processed image data.

[0091] In this embodiment, the enhanced division module 802 includes: an enhancement unit 8021, which is used to use an image processing library to perform data enhancement processing on historical preprocessed image data based on data enhancement technology to obtain enhanced historical image data; a removal unit 8022, which is used to remove enhanced historical images with abnormal markings from the enhanced historical image data to obtain historical image data to be divided; a division unit 8023, which is used to use a random sampling method to divide the historical image data to be divided into a training set, a verification set and a test set based on a preset division ratio.

[0092] In this embodiment, the construction module 803 includes: a construction unit 8031, which is used to construct a multi-task learning framework combining regression tasks and segmentation tasks, and adopts CSPDarknet53 as the backbone of the multi-task learning framework; a setting unit 8032, which is used to set the head network of the regression task and the head network of the segmentation task respectively; a definition unit 8033, which is used to define the mean square error function as the loss function of the regression task, define the cross entropy loss function as the loss function of the segmentation task, and define the comprehensive loss function of the regression task and the segmentation task to complete the construction of the task learning model.

[0093] In this embodiment, the construction module 803 also includes: a first design unit 8034, which is used to design a fully connected layer for implementing a regression task, and add a linear activation function after the fully connected layer to complete the setting of the head network for the regression task; a second design unit 8035, which is used to design a convolutional neural network structure for implementing a segmentation task, and the convolutional neural network structure includes multiple convolutional layers, activation layers and pooling layers; an embedding unit 8036, which is used to embed an upsampling technology for restoring the spatial resolution of the image at the output end of the pooling layer; an additional unit 8037, which is used to add a 1x1 convolutional layer for pixel-level classification after the pooling layer to perform pixel-level classification, and complete the setting of the head network for the segmentation task.

[0094] In this embodiment, the training and testing module 804 includes: a training unit 8041, which is used to input the training set into the constructed multi-task learning model for training, and use the verification set to perform performance evaluation on the multi-task learning model in the training process; an adjustment unit 8042, which is used to adjust the hyperparameters of the multi-task learning model based on the performance evaluation results of the verification set, combined with the early stopping method and the Bayesian optimization method, to obtain the model to be tested; a testing unit 8043, which inputs the test set into the model to be tested, and based on the test results, optimizes the model to be tested in combination with the regularization technology and the Bayesian optimization method to obtain a text detection model.

[0095] In this embodiment, the second processing module 805 includes: a second conversion unit 8051, which is used to obtain real-time waybill image data, perform format conversion processing on the real-time waybill data, and obtain standardized real-time waybill data; a second processing unit 8052, which is used to perform denoising processing, binarization processing and image correction processing on the standardized real-time waybill data, and obtain corrected real-time waybill data; a second positioning unit 8053, which is used to use image processing technology to perform text area positioning processing on the corrected real-time waybill data, and perform positioning marking to obtain real-time pre-processed image data.

[0096] above Figure 8The face sheet text detection device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the face sheet text detection device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0097] Fig. 9 1 is a schematic diagram of the structure of a bill text detection device provided by an embodiment of the present invention. The bill text detection device 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 910 (for example, one or more processors) and a memory 920, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 933 or data 932. Among them, the memory 920 and the storage medium 930 can be short-term storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the bill text detection device 900. Furthermore, the processor 910 can be configured to communicate with the storage medium 930, and execute a series of instruction operations in the storage medium 930 on the bill text detection device 900 to implement the steps of the bill text detection method provided by the above-mentioned method embodiments.

[0098] The document text detection device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input and output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Fig. 9 The structure of the sheet text detection device shown does not constitute a limitation on the sheet text detection device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0099] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the face sheet text detection method.

[0100] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0102] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting text on a bill, characterized in that: include: Acquire historical waybill image data, and preprocess the historical waybill image data to obtain historical preprocessed image data; Perform data enhancement and partitioning on the historical preprocessed image data to obtain a training set, a validation set, and a test set; Build a multi-task learning framework, and build a multi-task learning model based on the constructed multi-task learning framework; The training set and the validation set are respectively input into the constructed multi-task learning model for training, and the trained multi-task learning model is tested with the test set to obtain a text detection model; Acquire real-time waybill image data, and preprocess the real-time waybill image data to obtain real-time preprocessed image data; The real-time preprocessed image data is input into the text detection model to obtain the face sheet text detection result.

2. The method for detecting text on a label according to claim 1, characterized in that: The obtaining of historical waybill image data and preprocessing of the historical waybill image data to obtain historical preprocessed image data includes: Acquire historical mail order image data, convert multiple historical mail order images included in the historical mail order image data into the same format, and obtain multiple standardized historical mail order images; The standardized historical face order images are subjected to denoising, binarization and image correction processing respectively, to obtain multiple corrected historical face order images; Image processing technology is used to locate and mark the text areas in the corrected historical face image one by one, thereby obtaining multiple pre-processed historical images; Integrate multiple pre-processed historical images to obtain historical pre-processed image data.

3. The method for detecting text on a label according to claim 2, characterized in that: The historical pre-processed image data is subjected to data enhancement processing and division processing respectively to obtain a training set, a verification set and a test set, including: Using the image processing library, the historical pre-processed image data is enhanced based on the data enhancement technology to obtain enhanced historical image data; Eliminating enhanced historical images with abnormal markings from enhanced historical image data to obtain historical image data to be divided; Using a random sampling method, the historical image data to be divided is divided into a training set, a validation set, and a test set based on a preset division ratio.

4. The method for detecting text on a label according to claim 1, characterized in that: The multi-task learning framework is constructed, and a multi-task learning model is constructed based on the constructed multi-task learning framework, including: Construct a multi-task learning framework that combines regression tasks and segmentation tasks, and use CSPDarknet53 as the backbone of the multi-task learning framework; Set up the head network for the regression task and the head network for the segmentation task respectively; Define the mean square error function as the loss function of the regression task, define the cross entropy loss function as the loss function of the segmentation task, and define the comprehensive loss function of the regression task and the segmentation task to complete the construction of the learning model for all tasks.

5. The method for detecting text on a label according to claim 4, characterized in that: The head network of the regression task and the head network of the segmentation task are respectively set up, including: Design a fully connected layer for regression tasks, and add a linear activation function after the fully connected layer to complete the setting of the head network for regression tasks; Designing a convolutional neural network structure for implementing a segmentation task, wherein the convolutional neural network structure includes a plurality of convolutional layers, activation layers, and pooling layers; At the output of the pooling layer, an upsampling technique is embedded to restore the spatial resolution of the image; After the pooling layer, a 1x1 convolutional layer is added for pixel-level classification to complete the setting of the head network for the segmentation task.

6. The method for detecting text on a label according to claim 1, characterized in that: The training set and the validation set are respectively input into the constructed multi-task learning model for training, and the trained multi-task learning model is tested with the test set to obtain a text detection model, including: The training set is input into the constructed multi-task learning model for training, and the validation set is used to evaluate the performance of the multi-task learning model in the training process; Based on the performance evaluation results of the validation set, the hyperparameters of the multi-task learning model are adjusted by combining the early stopping method and the Bayesian optimization method to obtain the model to be tested; The test set is input into the model to be tested. Based on the test results, the model to be tested is optimized by combining regularization technology and Bayesian optimization method to obtain a text detection model.

7. The method for detecting text on a label according to claim 1, characterized in that: The step of acquiring real-time waybill image data and preprocessing the real-time waybill image data to obtain real-time preprocessed image data includes: Acquire real-time waybill image data, perform format conversion on the real-time waybill data, and obtain standardized real-time waybill data; The standardized real-time waybill data is subjected to denoising, binarization and image correction respectively to obtain corrected real-time waybill data; Image processing technology is used to locate the text area of ​​the corrected real-time face sheet data, and positioning marking is performed to obtain real-time pre-processed image data.

8. A device for detecting text on a bill, characterized in that: include: The first processing module is used to obtain historical waybill image data and preprocess the historical waybill image data to obtain historical preprocessed image data; An enhancement and partitioning module is used to perform data enhancement and partitioning processing on the historical preprocessed image data to obtain a training set, a validation set, and a test set; A construction module, used to construct a multi-task learning framework, and to construct a multi-task learning model based on the constructed multi-task learning framework; The training and testing module is used to input the training set and the validation set into the constructed multi-task learning model for training, and use the test set to test the trained multi-task learning model to obtain a text detection model; The second processing module is used to obtain real-time waybill image data and pre-process the real-time waybill image data to obtain real-time pre-processed image data; The detection module is used to input the real-time pre-processed image data into the text detection model to obtain the face sheet text detection result.

9. A document text detection device, characterized in that: The bill text detection device comprises: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors calls the instructions in the memory so that the face sheet text detection device executes each step of the face sheet text detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the method for detecting text on a receipt as described in any one of claims 1 to 7 are implemented.

Images