Method, device, electronic device and storage medium for voucher classification

Abstract

The present invention discloses a method, device, electronic device and storage medium for voucher classification, and relates to the field of artificial intelligence technology. A specific implementation of the method includes: obtaining a voucher image in a voucher processing request; calling an image recognition model to identify background information in the voucher image to query the corresponding business type; calling a semantic segmentation model corresponding to the business type to calculate the text position information corresponding to each element parameter in the voucher image to segment and obtain the text image corresponding to each element parameter; calling a text recognition model to identify the text information of each text image, obtain the text information set of the voucher image, and then determine the feature vector of the text information set; calling a classification model to determine the type of the voucher image, and sending the type of the voucher image to the terminal. This implementation can solve the problem that the existing recognition method has difficulty in identifying non-standard vouchers, resulting in the inability to accurately identify the type of voucher, thereby reducing the accuracy of voucher classification.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for credential classification. Background Art

[0002] In business scenarios using standardized vouchers, the existing technology usually uses template recognition or QR code recognition to identify the type of voucher and then classify the voucher. However, for some non-standard vouchers that do not use a standardized format or are printed independently by other institutions, the fonts, languages, and text positions used are diverse. The existing recognition methods have difficulty in identifying their contents, resulting in the inability to accurately identify the type of voucher, thereby reducing the accuracy of voucher classification. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a method, device, electronic device and storage medium for credential classification, which can solve the problem that existing identification methods have difficulty in identifying non-standard credentials, resulting in the inability to accurately identify the type of credentials, thereby reducing the accuracy of credential classification.

[0004] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for voucher classification is provided.

[0005] A method for voucher classification in an embodiment of the present invention includes: receiving a voucher processing request sent by a terminal, and obtaining a voucher image in the voucher processing request; calling an image recognition model to identify background information in the voucher image to query the corresponding business type based on a preset mapping relationship; calling a semantic segmentation model corresponding to the business type to calculate text position information corresponding to each element parameter in the voucher image to segment and obtain text images corresponding to each element parameter; calling a text recognition model to identify text information of each text image to obtain a text information set of the voucher image, and then calculating the word frequency feature value of each text information in the text information set to determine a feature vector of the text information set; based on the feature vector, calling a classification model to determine the type of the voucher image, and sending the type of the voucher image to the terminal.

[0006] In one embodiment, the calling of the image recognition model to identify the background information in the voucher image to query the corresponding business type based on a preset mapping relationship includes:

[0007] The image recognition model is called to recognize characters included in the background image in the voucher image, and based on a preset mapping relationship, the business type corresponding to the characters is queried from a database.

[0008] In yet another embodiment, the characters included in the background image in the credential image include character patterns and / or character strings.

[0009] In yet another embodiment, after calling the text recognition model to recognize the text information of the text image and obtaining the text information set, the method further includes:

[0010] Based on the correspondence between the element parameters and the text image, determining the text information corresponding to the element parameters;

[0011] Sending text information corresponding to the service element parameters to the terminal.

[0012] In yet another embodiment, the calculating of the word frequency feature value of each text information in the text information set includes:

[0013] For each text information in the text information set, the number of times each text information appears in the text set and the inverse document frequency of each text information are calculated, and the product of the number and the inverse document frequency is determined as the word frequency feature value of each text information.

[0014] In yet another embodiment, before receiving the classification request sent by the terminal, the method further includes:

[0015] Acquire historical voucher images including type labels and models to be trained, wherein the models to be trained include image recognition models, semantic segmentation models, text recognition models, and classification models to be trained;

[0016] Receiving an input instruction, obtaining a pending parameter value in the input instruction, and configuring the parameter to be adjusted in the model to be trained to the pending parameter value;

[0017] The historical voucher image is input into the model to be trained to adjust the parameter value to be determined, thereby obtaining a trained model.

[0018] In yet another embodiment, obtaining the historical credential image including the type tag includes:

[0019] Acquire multiple historical credential images from a historical credential image library;

[0020] A preset annotation model is called to annotate the multiple historical voucher images with type labels of the types they belong to, thereby obtaining the historical voucher images including the type labels.

[0021] In yet another embodiment, after acquiring a plurality of historical credential images from the historical credential image library, the method further includes:

[0022] A preset annotation model is called to annotate the plurality of historical voucher images with labels of text positions corresponding to element parameters.

[0023] To achieve the above objective, according to another aspect of an embodiment of the present invention, a device for classifying vouchers is provided.

[0024] A device for voucher classification in an embodiment of the present invention includes: a receiving unit, which is used to receive a voucher processing request sent by a terminal and obtain a voucher image in the voucher processing request; a query unit, which is used to call an image recognition model to identify background information in the voucher image so as to query the corresponding business type based on a preset mapping relationship; a segmentation unit, which is used to call a semantic segmentation model corresponding to the business type and calculate text position information corresponding to each element parameter in the voucher image so as to segment and obtain text images corresponding to each element parameter; a recognition unit, which is used to call a text recognition model to identify text information of each text image and obtain a text information set of the voucher image, and then calculate a word frequency feature value of each text information in the text information set to determine a feature vector of the text information set; and a classification unit, which is used to call a classification model to determine the type of the voucher image according to the feature vector and send the type of the voucher image to the terminal.

[0025] In one embodiment, the query unit is specifically used to:

[0026] The image recognition model is called to recognize characters included in the background image in the voucher image, and based on a preset mapping relationship, the business type corresponding to the characters is queried from a database.

[0027] In yet another embodiment, the apparatus comprises:

[0028] A determination unit, configured to determine text information corresponding to the element parameter based on a correspondence between the element parameter and the text image;

[0029] A sending unit is used to send text information corresponding to the service element parameters to the terminal.

[0030] In yet another embodiment, the characters included in the background image in the credential image include character patterns and / or character strings.

[0031] In yet another embodiment, the identification unit is specifically configured to:

[0032] For each text information in the text information set, the number of times each text information appears in the text set and the inverse document frequency of each text information are calculated, and the product of the number and the inverse document frequency is determined as the word frequency feature value of each text information.

[0033] In yet another embodiment, the apparatus further comprises:

[0034] An acquisition unit, used to acquire a historical voucher image including a type label and a model to be trained, wherein the model to be trained includes an image recognition model, a semantic segmentation model, a text recognition model, and a classification model to be trained;

[0035] A configuration unit, configured to receive an input instruction, obtain a pending parameter value in the input instruction, and configure the parameter to be adjusted in the model to be trained to the pending parameter value;

[0036] An adjustment unit is used to input the historical voucher image into the model to be trained to adjust the parameter value to be determined and obtain a trained model.

[0037] In yet another embodiment, the acquisition unit is specifically configured to:

[0038] Acquire multiple historical credential images from a historical credential image library;

[0039] A preset annotation model is called to annotate the multiple historical voucher images with type labels of the types they belong to, thereby obtaining the historical voucher images including the type labels.

[0040] In yet another embodiment, the acquisition unit is specifically configured to:

[0041] A preset annotation model is called to annotate the plurality of historical voucher images with labels of text positions corresponding to element parameters.

[0042] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0043] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method for credential classification provided by the embodiment of the present invention.

[0044] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer readable medium is provided.

[0045] A computer-readable medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements the method for classifying vouchers provided by an embodiment of the present invention.

[0046] One embodiment of the above invention has the following advantages or beneficial effects: In the embodiment of the present invention, for the voucher image to be classified, the background information can be first identified to determine the business type corresponding to the background information, and then the text information of each element parameter in the voucher image can be identified based on the semantic segmentation model and the text recognition model corresponding to the business type, and the word frequency feature value corresponding to each text information can be calculated, and then the feature vector can be determined, and the type of the voucher image can be determined by inputting the feature vector into the classification model. In the embodiment of the present invention, for each voucher image, the business type can be determined based on the background information, and then the text information of each element parameter can be identified by the text recognition model, so that the type of the voucher image can be determined by the text information of each element and the classification model, so that the type of non-standard vouchers can be accurately identified, and the accuracy of voucher classification can be improved.

[0047] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0049] Figure 1 is a schematic diagram of a main process of a method for voucher classification according to an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a text image corresponding to element parameters according to an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of a main process of a model training method according to an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of a main process of a method for labeling a historical voucher image according to an embodiment of the present invention;

[0053] Figure 5 is a schematic diagram of a main process of a model training and deployment method according to an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of main units of an apparatus for credential classification according to an embodiment of the present invention;

[0055] Figure 7 is another exemplary system architecture diagram to which the embodiments of the present invention can be applied;

[0056] Figure 8 It is a schematic diagram of the structure of a computer system suitable for implementing the embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0058] It should be pointed out that the embodiments of the present invention and the features in the embodiments may be combined with each other if there is no conflict.

[0059] The embodiment of the present invention provides a system for credential classification, which can be used in a scenario of classifying credential images. Specifically, the system for credential classification can include a terminal and a server, and the terminal can send a credential image to be classified to the server. After receiving the credential image, the server can process it, determine the type of the credential image, and send the determined type to the terminal.

[0060] The embodiment of the present invention provides a method for classifying credentials, which can be executed by a server. Figure 1 As shown, the method includes the following steps.

[0061] S101: receiving a credential processing request sent by a terminal, and acquiring a credential image in the credential processing request.

[0062] The terminal sends a credential processing request to the server, and the credential processing request includes a credential image that needs to be classified. After receiving the credential processing request, the server can obtain the credential image from it.

[0063] S102: calling an image recognition model to identify background information in the voucher image, so as to query the corresponding business type based on a preset mapping relationship.

[0064] Among them, the voucher image usually includes a background image and a target image. The target image can be each element parameter in the voucher and the parameter value of the element parameter. For example, when the voucher is a contract, the element parameter can include the contract name, contract number, buyer, seller, etc. The background image can be other content in the voucher image except the target image, such as the abbreviation mark of some companies, the general icon of some industries, etc. These contents are just a mark and are usually located on the side of the voucher image, but they can represent the industry type to which the voucher belongs. Therefore, in this step, the image recognition model can be trained, and the target image and background image in the voucher representation can be identified by the image recognition model, and then the background image can be segmented to extract the background information. In the embodiment of the present invention, the mapping relationship between each background information and the business type is pre-set, so the business type corresponding to the background information can be queried based on the preset mapping relationship. In the embodiment of the present invention, the background information can be specifically included as characters, such as character patterns and / strings, for example, the logo of a company, abbreviation characters, etc., and the preset mapping relationship is stored in the database, so this step can be specifically performed as follows: calling the image recognition model, identifying the characters included in the background image in the voucher image, and querying the business type corresponding to the characters from the database based on the preset mapping relationship.

[0065] It should be noted that the business type can be divided based on specific scenarios, for example, the business type can include catering, finance, industry, etc. For some vouchers without background information, the image recognition model in this step cannot identify the background information or identifies that the background information is empty, then the business type can be determined to be a general type.

[0066] S103: calling a semantic segmentation model corresponding to the business type, calculating text position information corresponding to each element parameter in the voucher image, and segmenting to obtain text images corresponding to each element parameter.

[0067] Among them, due to different business types, the corresponding vouchers may be different, so in the embodiment of the present invention, the vouchers of all business types use the same semantic segmentation model, which will bring great difficulty to the model training, so in the embodiment of the present invention, the corresponding segmentation model can be trained for each business type to reduce the amount of model training. In this step, the corresponding semantic segmentation model can be determined based on the business type, and the text position information corresponding to each element parameter of the voucher image can be calculated by the semantic segmentation model. The text position information may include the element parameter name and the element parameter value in the voucher image, so the text position information obtained in this step may include the position information of the element parameter name and the element parameter value. For example, if the element parameter is a date and the element parameter value is January 1, 200, the text position information includes the text position information of both the date and January 1, 200. The text position information can be specifically the coordinates of the rectangular box where the text is located.

[0068] This step is subsequently classified based on the text information corresponding to the element parameters in the voucher, so after determining the text position information, the text image corresponding to each element parameter can be segmented from the voucher image.

[0069] Since the voucher image may have factors such as diverse fonts, diverse languages, variable positional relationships, and complex backgrounds that affect recognition, this step can use a region-based semantic segmentation method. For example, the R-CNN algorithm model can be used. After training, it can use selective search to extract a large number of texts corresponding to element parameters, then calculate the CNN features of each text, and then use a class-specific linear support vector machine to classify each region, and finally output the text image corresponding to the element parameters in the voucher image.

[0070] It should be noted that in the embodiment of the present invention, a semantic segmentation model applicable to various business fields can also be trained to facilitate processing of general-type voucher images for business types and segmentation to obtain text images corresponding to various element parameters in the voucher images.

[0071] S104: calling a text recognition model to recognize text information of each text image, obtaining a text information set of the voucher image, and then calculating a word frequency feature value of each text information in the text information set to determine a feature vector of the text information set.

[0072] Among them, after obtaining the text image of the element parameters in the voucher image, the text recognition model can be called to recognize the text in the text image, so as to obtain the text information of each text image, and the text information of each text image is combined to obtain the text information set of each element parameter, that is, the text information set of the voucher image.

[0073] Specifically, in this step, the recognition of text images can use the encode-decode framework in deep learning, the AutoML model, the neural architecture search NAS, and the adaptive receptive field network to build a text recognition model. Among them, the text recognition model is based on the Encode-Decode technology combined with the Auto-ML technology, and uses the NAS technology based on the adaptive receptive field to build a neural network. The network structure of each layer is the "appropriate receptive field" substructure searched by NAS, and the information around the characters in the text is used to assist in the recognition of the characters. The amount of information needed is obtained by NAS through learning. It also supports dynamic adjustment of the amount of calculation, which can ensure the flexible deployment of the model in various scenarios such as the cloud and local terminals.

[0074] like Figure 2The figure shows a schematic diagram of a text image. The text in the text image is the contract agreement number. After being processed by the text recognition model, the text information corresponding to the text image, namely the contract / invoice number, can be obtained.

[0075] After the text information set is obtained in this step, it can be converted into a bag-of-words model. At this time, the lexical structure and word order of the text information are not considered. Each text information can be used as an independent word, that is, a feature. Each text information in the text information set can be mapped into a dictionary array. In this way, the feature vector of the text information set can be determined by calculating the word frequency feature value of each text information.

[0076] Specifically, in the embodiment of the present invention, the word frequency feature value can be a word value, and the word value of each text information in the text information set can be calculated based on the TF-IDF model. Therefore, in this step, the step of calculating the word frequency feature value of each text information in the text information set can be specifically performed as follows: for each text information in the text information set, the number of times each text information appears in the text set and the inverse document frequency of each text information are calculated, and the product of the number and the inverse document frequency is determined as the word frequency feature value of each text information.

[0077] The value of a word can be expressed as the frequency of the word appearing in the text where the word is located multiplied by the inverse document frequency of the word. For words that do not appear in the text, the corresponding value is 0. In this step, for each text information in the text information set, the number of times the text information appears in the text set and the inverse document frequency of each text information can be calculated first, and then the product of the number and the inverse document frequency is determined as the word frequency feature value of the text information. Specifically, the word frequency feature value of the text information can be calculated by Formula 1.

[0078] w j = count(t j )×idf(t j ), the text information set includes text information t j (1)

[0079]

[0080]

[0081] In formula 1, w j Represents the jth text information t in the text information set j The word frequency feature value, count(t j ) indicates that the text information set contains t j The number of text information, idf(t j ) indicates t j The inverse document frequency, formula 2 is idf(t j), Formula 3 is the non-smooth calculation formula of idf(t j ) is the smoothing calculation formula. In formula 1, idf(t j ) can be calculated by Formula 2 or Formula 3, and which formula to use can be selected based on the specific scenario.

[0082] It should be noted that when there are many credential images, the word frequency feature value of the text information in the text information set corresponding to each text image can be calculated through a distributed bag-of-words model. Specifically, in the embodiment of the present invention, multiple devices or equipment for calculating word frequency feature values ​​can be set, and the server can distribute the text information set of each credential image to each device or equipment for calculating word frequency feature values ​​based on the identification of the credential image, and receive the calculation results returned by each device or equipment after the calculation is completed, that is, the word frequency feature value of each text information in the text information set.

[0083] After calculating the word frequency feature value of each text information in the text set, it can be mapped to the vector space to obtain the feature vector of the text information set. Specifically, in this step, the word frequency feature value of each text information can be normalized, and then the processed values ​​are arranged in sequence to obtain the feature vector.

[0084] S105: According to the feature vector, a classification model is called to determine the type of the credential image, and the type of the credential image is sent to the terminal.

[0085] After the feature vector is obtained in this step, the classification model can be called, the feature vector is input into the classification model to calculate the type of the credential image, and the calculation result is sent to the terminal to notify the terminal of the type of the credential image. Specifically, in an embodiment of the present invention, the classification model can be a model constructed by BERT+fune turn.

[0086] It should be noted that, in the embodiment of the present invention, after processing the credential image, not only the type of the credential image can be obtained, but also a text information set can be obtained in step S104, that is, the text information corresponding to the element parameters. Therefore, after the text information set is obtained in step S104, it is also possible to execute: based on the correspondence between the element parameters and the text image, determine the text information corresponding to the element parameters; send the text information corresponding to the business element parameters to the terminal.

[0087] Since the element parameters correspond to the text image, and the text image corresponds to the text information, the text information corresponding to the element parameters can be determined, and then the text information corresponding to the element parameters is sent to the terminal so that the terminal can determine the content in the voucher. In this step, the text information corresponding to the element parameters and the type of the voucher image can be sent to the terminal at the same time.

[0088] In an embodiment of the present invention, for each credential image, the business type can be first determined based on the background information, and then the text information of each element parameter can be identified through a text recognition model. In this way, the type of the credential image can be determined through the text information of each element and a classification model, thereby achieving accurate identification of the type of non-standard credentials and improving the accuracy of credential classification.

[0089] Combine the following Figure 1 In the embodiment shown in the figure, various models are used in the embodiment of the present invention, so it is necessary to perform Figure 1 Before the embodiment shown, each model is trained, and the method of model training in the embodiment of the present invention is specifically described. Figure 3 As shown, the method includes:

[0090] S301: Acquire historical voucher images including type labels and a model to be trained.

[0091] The models to be trained include image recognition models, semantic segmentation models, text recognition models and classification models to be trained. The models to be trained in this step can be various models used in the embodiments of the present invention, or the models used can be integrated into one model to be trained according to the execution process.

[0092] In the embodiment of the present invention, annotated historical voucher images are required for model training, so the historical voucher images need to be annotated first. Specifically, multiple historical voucher images can be obtained from the historical voucher image library, and then a preset annotation model can be called to annotate the multiple historical voucher images with type labels of the types they belong to, so as to obtain historical voucher images including type labels. This can be achieved by using an annotation model or by manual annotation. The annotation content mainly includes the type of the historical voucher image, the text position corresponding to the element parameters in the voucher image, such as the regional coordinates and text information of each element parameter and the text corresponding to the element parameter.

[0093] As shown in Table 1, these are the element parameters corresponding to different types of credentials.

[0094] Table 1

[0095]

[0096]

[0097] Specifically, the marking process of historical vouchers can be as follows: Figure 4 As shown, Figure 4The image in the image represents a historical voucher image. When annotating, the annotator needs to annotate the parameters of each element in the voucher image, and the annotation needs to be accurate, so the content in the voucher needs to be read and transcribed, which places extremely high demands on the annotator's concentration under long-term multi-tasking. In order to reduce the workload of the annotators, the embodiments of the present invention can apply some existing models to perform pre-processing operations such as regional segmentation, text recognition, and natural language understanding on the historical voucher images. After the pre-annotation is automatically completed, the annotator will manually proofread and annotate to obtain a set of historical voucher images with labels, that is, a labeled data set, thereby improving the annotation efficiency while effectively reducing the annotation cost. In an embodiment of the present invention, the annotation of historical voucher images can be achieved through an active learning model.

[0098] S302: receiving an input instruction, obtaining a pending parameter value in the input instruction, and configuring the parameter to be adjusted in the model to be trained as the pending parameter value.

[0099] Among them, for each model to be trained, it is mainly to adjust the parameter value in the model through the training process, so as to determine the parameter value that enables the model to obtain accurate results. In this step, each model can be first configured with a parameter value close to the parameter value that enables the model to obtain accurate results, that is, the parameter value to be determined, to reduce the training process. Therefore, an input instruction can be received, and the input instruction is usually an input instruction sent by a user, and the input instruction includes the parameter value to be determined, and then the parameter to be adjusted in the model to be trained can be configured as the parameter value to be determined, so as to perform the subsequent training process.

[0100] S303: Inputting the historical voucher image into the model to be trained to adjust the parameter values ​​to be determined and obtain the trained model.

[0101] The training of the model is to input the historical voucher images into the model to be trained, adjust the parameter values ​​through the continuous iterative cycle training process, and test the trained model through test data to determine whether the accuracy of the model output results meets the preset conditions, until a usable model is obtained, and the trained model can be obtained, that is, Figure 1 The models used in the illustrated embodiments.

[0102] After the model training is completed, it is usually necessary to deploy it, and it can be used after deployment. Figure 5 The figure is a schematic diagram of a method from training to deployment of a model to be trained in an embodiment of the present invention.

[0103] Among them, the labeled data set represents the annotated historical voucher images, the domain base model represents the model to be trained, the domain data represents some algorithms used in the model to be trained, such as finetune, the model parameter adjustment represents the process of adjusting parameters in step 302 and step S303 in the embodiment of the present invention, the model training represents the process of training in step S303, and the model verification represents the process of verifying the model through test data and preset conditions after the model is trained. After the model is verified, it can be deployed through model pruning and model quantization.

[0104] In the embodiment of the present invention, each model for identifying the voucher is obtained by training each model to be trained, so that the type of the voucher image can be accurately determined.

[0105] When there are vouchers with various fonts, languages, positional relationships and complex backgrounds, the combination of various models can derive highly accurate structural element parameters and text information corresponding to the element parameters. When classifying the types of vouchers, each bill is abstracted into a bag-of-words model in the embodiment of the present invention, and the feature vector of the text information set can be obtained in a distributed bag-of-words mode, and finally the type of non-standard voucher image is accurately determined through the classification model.

[0106] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a device 600 for classifying vouchers, such as Figure 6 As shown, the device 600 includes:

[0107] The receiving unit 601 is configured to receive a credential processing request sent by a terminal, and obtain a credential image in the credential processing request;

[0108] A query unit 602 is used to call an image recognition model to identify background information in the voucher image to query the corresponding business type based on a preset mapping relationship;

[0109] The segmentation unit 603 is used to call the semantic segmentation model corresponding to the business type, calculate the text position information corresponding to each element parameter in the voucher image, and segment and obtain the text image corresponding to each element parameter;

[0110] The recognition unit 604 is used to call the text recognition model to recognize the text information of each text image, obtain the text information set of the voucher image, and then calculate the word frequency feature value of each text information in the text information set to determine the feature vector of the text information set;

[0111] The classification unit 605 is configured to call a classification model to determine the type of the credential image according to the feature vector, and send the type of the credential image to the terminal.

[0112] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 1 The method of the illustrated embodiment is the same and will not be repeated here.

[0113] In an implementation of the embodiment of the present invention, the query unit 602 is specifically configured to:

[0114] The image recognition model is called to recognize characters included in the background image in the voucher image, and based on a preset mapping relationship, the business type corresponding to the characters is queried from a database.

[0115] In yet another implementation of the embodiment of the present invention, the characters included in the background image in the credential image include character patterns and / or character strings.

[0116] In another implementation of the embodiment of the present invention, the device 600 includes:

[0117] A determination unit, configured to determine text information corresponding to the element parameter based on a correspondence between the element parameter and the text image;

[0118] A sending unit is used to send text information corresponding to the service element parameters to the terminal.

[0119] In another implementation of the embodiment of the present invention, the identification unit 604 is specifically configured to:

[0120] For each text information in the text information set, the number of times each text information appears in the text set and the inverse document frequency of each text information are calculated, and the product of the number and the inverse document frequency is determined as the word frequency feature value of each text information.

[0121] In another implementation of the embodiment of the present invention, the device 600 further includes:

[0122] An acquisition unit, used to acquire a historical voucher image including a type label and a model to be trained, wherein the model to be trained includes an image recognition model, a semantic segmentation model, a text recognition model, and a classification model to be trained;

[0123] A configuration unit, configured to receive an input instruction, obtain a pending parameter value in the input instruction, and configure the parameter to be adjusted in the model to be trained to the pending parameter value;

[0124] An adjustment unit is used to input the historical voucher image into the model to be trained to adjust the parameter value to be determined and obtain a trained model.

[0125] In another implementation of the embodiment of the present invention, the acquisition unit is specifically configured to:

[0126] Acquire multiple historical credential images from a historical credential image library;

[0127] Calling a preset annotation model to annotate the multiple historical voucher images with type labels of the types they belong to, and obtaining the historical voucher images including the type labels

[0128] In another implementation of the embodiment of the present invention, the acquisition unit is specifically configured to:

[0129] A preset annotation model is called to annotate the plurality of historical voucher images with labels of text positions corresponding to element parameters.

[0130] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 2 The method of the illustrated embodiment is the same and will not be repeated here.

[0131] In an embodiment of the present invention, for each credential image, the business type can be first determined based on the background information to determine the element parameters to be identified in the credential, and then the text information of each element parameter can be identified through a text recognition model. In this way, the type of the credential image can be determined through the text information of each element and the classification model, thereby achieving accurate identification of the type of non-standard credentials and improving the accuracy of credential classification.

[0132] According to an embodiment of the present invention, the embodiment of the present invention further provides an electronic device and a readable storage medium.

[0133] The electronic device of an embodiment of the present invention includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method for credential classification provided by the embodiment of the present invention.

[0134] Figure 7 An exemplary system architecture 700 is shown to which the method for credential classification or the apparatus for credential classification according to an embodiment of the present invention can be applied.

[0135] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0136] The user can use the terminal devices 701, 702, 703 to interact with the server 705 through the network 704 to receive or send messages, etc. Various client applications can be installed on the terminal devices 701, 702, 703.

[0137] The terminal devices 701 , 702 , and 703 may be, but are not limited to, smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0138] The server 705 may be a server that provides various services. The server may analyze and process the received data such as the product information query request, and feed back the processing result (such as product information - just an example) to the terminal device.

[0139] It should be noted that the method for credential classification provided in the embodiment of the present invention is generally executed by the server 705 , and accordingly, the device for credential classification is generally set in the server 705 .

[0140] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0141] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 suitable for implementing an embodiment of the present invention. Figure 8 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0142] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0143] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0144] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0145] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0146] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a unit, a program segment, or a part of a code, and the above-mentioned unit, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be arranged in a processor, for example, it may be described as: a processor includes a receiving unit, a query unit, a segmentation unit, an identification unit, and a classification unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the receiving unit may also be described as a "unit of the function of the receiving unit".

[0148] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes the method for voucher classification provided by the present invention.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for voucher classification, characterized in that: include: Receiving a credential processing request sent by a terminal, and acquiring a credential image in the credential processing request; Calling an image recognition model to identify background information in the voucher image, so as to query the corresponding business type based on a preset mapping relationship; Calling the semantic segmentation model corresponding to the business type, calculating the text position information corresponding to each element parameter in the voucher image, so as to segment and obtain the text image corresponding to each element parameter; Calling a text recognition model to recognize text information of each text image, obtaining a text information set of the voucher image, and then calculating a word frequency feature value of each text information in the text information set to determine a feature vector of the text information set; calling a classification model to determine the type of the credential image according to the feature vector, and sending the type of the credential image to the terminal; The calling of the image recognition model to identify the background information in the voucher image to query the corresponding business type based on a preset mapping relationship includes: Calling an image recognition model to recognize characters included in the background image in the voucher image, and querying a database for a business type corresponding to the characters based on a preset mapping relationship; When the background information cannot be identified through the image recognition model or the identified background information is empty, the business type is determined to be a general type.

2. The method according to claim 1, characterized in that The characters included in the background image in the voucher image include character patterns and / or character strings.

3. The method according to claim 1, characterized in that After calling the text recognition model to recognize the text information of the text image and obtaining the text information set, the method further includes: Based on the correspondence between the element parameters and the text image, determining the text information corresponding to the element parameters; Sending text information corresponding to the element parameter to the terminal.

4. The method according to claim 1, characterized in that The calculating of the word frequency feature value of each text information in the text information set includes: For each text information in the text information set, the number of times each text information appears in the text set and the inverse document frequency of each text information are calculated, and the product of the number and the inverse document frequency is determined as the word frequency feature value of each text information.

5. The method according to claim 1, characterized in that Before the receiving terminal sends a classification request, the method further includes: Acquire historical voucher images including type labels and models to be trained, wherein the models to be trained include image recognition models, semantic segmentation models, text recognition models, and classification models to be trained; Receiving an input instruction, obtaining a pending parameter value in the input instruction, and configuring the parameter to be adjusted in the model to be trained to the pending parameter value; The historical voucher image is input into the model to be trained to adjust the parameter value to be determined, thereby obtaining a trained model.

6. The method according to claim 5, characterized in that The obtaining of the historical credential image including the type tag comprises: Acquire multiple historical credential images from a historical credential image library; A preset annotation model is called to annotate the multiple historical voucher images with type labels of the types they belong to, thereby obtaining the historical voucher images including the type labels.

7. The method according to claim 6, characterized in that After obtaining a plurality of historical credential images from the historical credential image library, the method further includes: A preset annotation model is called to annotate the plurality of historical voucher images with labels of text positions corresponding to element parameters.

8. A device for voucher classification, characterized in that: include: A receiving unit, configured to receive a credential processing request sent by a terminal, and obtain a credential image in the credential processing request; A query unit, configured to call an image recognition model to identify background information in the voucher image, so as to query a corresponding business type based on a preset mapping relationship; A segmentation unit, used to call a semantic segmentation model corresponding to the business type, calculate text position information corresponding to each element parameter in the voucher image, and segment and obtain text images corresponding to each element parameter; A recognition unit, used to call a text recognition model to recognize text information of each text image, obtain a text information set of the voucher image, and then calculate a word frequency feature value of each text information in the text information set to determine a feature vector of the text information set; a classification unit, configured to call a classification model to determine the type of the credential image according to the feature vector, and send the type of the credential image to the terminal; The query unit is specifically used for: An image recognition model is called to identify characters included in the background image in the voucher image, and based on a preset mapping relationship, the business type corresponding to the characters is queried from a database; when the background information cannot be identified by the image recognition model or the background information is identified to be empty, the business type is determined to be a general type.

9. The device according to claim 8, characterized in that The device comprises: A determination unit, configured to determine text information corresponding to the element parameter based on a correspondence between the element parameter and the text image; A sending unit is used to send text information corresponding to the element parameter to the terminal.

10. The device according to claim 8, characterized in that The identification unit is specifically used for: For each text information in the text information set, the number of times each text information appears in the text set and the inverse document frequency of each text information are calculated, and the product of the number and the inverse document frequency is determined as the word frequency feature value of each text information.

11. The device according to claim 8, characterized in that The device also includes: An acquisition unit, used to acquire a historical voucher image including a type label and a model to be trained, wherein the model to be trained includes an image recognition model, a semantic segmentation model, a text recognition model, and a classification model to be trained; A configuration unit, configured to receive an input instruction, obtain a pending parameter value in the input instruction, and configure the parameter to be adjusted in the model to be trained to the pending parameter value; An adjustment unit is used to input the historical voucher image into the model to be trained to adjust the parameter value to be determined and obtain a trained model.

12. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

13. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images