Method, apparatus and device for detecting structured information

Abstract

The embodiment of the present application provides a kind of detection method, device and equipment of structured information.The method comprises: obtaining the file image of the file to be identified;Determine the file type of the file to be identified based on file image;The field attribute of field information in file image is obtained by carrying out structured detection to file image;According to file type and field attribute, determine the structured information included in the file to be identified.The technical scheme provided by the embodiment of the present application determines the file type of the file to be identified by the obtained file image, and carries out structured detection to file image, obtains the field attribute of field information in file image, and then determines the structured information included in the file to be identified according to file type and field attribute, effectively realizes the structured information detection operation of end to end based on image global semantic information, so that the quality and efficiency of detecting structured information are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information detection, and in particular to a structured information detection method, device and equipment. BACKGROUND

[0002] With the rapid development of science and technology, the application of cards, certificates and bills is becoming more and more widespread. In order to improve the quality and efficiency of the application of cards, certificates and bills, electronic application can be performed on various cards, certificates and bills. At this time, information recognition operation needs to be performed on various cards, certificates and bills. In the prior art, the information recognition operation is usually implemented by using optical character recognition (OCR) technology, which specifically includes the following steps: detecting the text of the card, certificate or bill by using a text detection model, and then extracting the relationship between the text blocks by using a graph convolutional network (GCN) combined with the text recognition content, so as to extract the structured information of the card, certificate or bill.

[0003] However, the above information detection method cannot realize end-to-end information extraction operation, and when identifying cards, certificates and bills, it can only train a special information detection model for a specific type of card, certificate or bill, and it is difficult to be universal. SUMMARY

[0004] The embodiments of the present application provide a structured information detection method, device and equipment, which can realize end-to-end structured information extraction operation and can identify the structured information of various cards, certificates and bills, thereby improving the application range and practicality of the method.

[0005] In a first aspect, the embodiments of the present application provide a structured information detection method, which includes:

[0006] obtaining a file image of a to-be-identified file;

[0007] determining a file type of the to-be-identified file based on the file image;

[0008] performing structured detection on the file image to obtain a field attribute of field information in the file image;

[0009] determining structured information included in the to-be-identified file according to the file type and the field attribute.

[0010] In a second aspect, the embodiments of the present application provide a structured information detection device, which includes:

[0011] a first obtaining module configured to obtain a file image of a to-be-identified file;

[0012] The first determining module is configured to determine the file type of the file to be recognized based on the file image.

[0013] The first detecting module is configured to perform structural detection on the file image to obtain field attributes of field information in the file image.

[0014] The first processing module is configured to determine the structured information included in the file to be recognized according to the file type and the field attributes.

[0015] In a third aspect, an electronic device is provided, including a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method for detecting structured information in the first aspect.

[0016] In a fourth aspect, a computer storage medium is provided for storing a computer program, and the computer program causes a computer to implement the method for detecting structured information in the first aspect.

[0017] In a fifth aspect, a method for detecting structured information is provided, including:

[0018] In response to a request for calling an information recognition service, determining a processing resource corresponding to the information recognition service;

[0019] Using the processing resource to perform the following steps: obtaining a file image of a file to be recognized; determining a file type of the file to be recognized based on the file image; performing structural detection on the file image to obtain field attributes of field information in the file image; and determining structured information included in the file to be recognized according to the file type and the field attributes.

[0020] In a sixth aspect, a device for detecting structured information is provided, including:

[0021] The second determining module is configured to determine a processing resource corresponding to the information recognition service in response to a request for calling the information recognition service;

[0022] The second processing module is configured to use the processing resource to perform the following steps: obtaining a file image of a file to be recognized; determining a file type of the file to be recognized based on the file image; performing structural detection on the file image to obtain field attributes of field information in the file image; and determining structured information included in the file to be recognized according to the file type and the field attributes.

[0023] In a seventh aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method for detecting structured information in the fifth aspect.

[0024] In an eighth aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, wherein the computer program causes a computer to implement the method for detecting structured information in the fifth aspect when executed.

[0025] In a ninth aspect, an embodiment of the present application provides a method for detecting structured information, comprising:

[0026] obtaining an information recognition request for a to-be-recognized image;

[0027] determining structured information included in the to-be-recognized image based on the information recognition request, wherein the structured information is determined based on a file type of the to-be-recognized image and field attributes of field information in the to-be-recognized image, and the field attributes of the field information in the to-be-recognized image are obtained by performing structured detection on the to-be-recognized image;

[0028] displaying the structured information included in the to-be-recognized image.

[0029] In a tenth aspect, an embodiment of the present application provides a device for detecting structured information, comprising:

[0030] a third obtaining module configured to obtain an information recognition request for a to-be-recognized image;

[0031] a third determining module configured to determine structured information included in the to-be-recognized image based on the information recognition request, wherein the structured information is determined based on a file type of the to-be-recognized image and field attributes of field information in the to-be-recognized image, and the field attributes of the field information in the to-be-recognized image are obtained by performing structured detection on the to-be-recognized image;

[0032] a third displaying module configured to display the structured information included in the to-be-recognized image.

[0033] In an eleventh aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method for detecting structured information in the ninth aspect.

[0034] In a twelfth aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, which, when executed by a computer, implements the structured information detection method in the ninth aspect.

[0035] The structured information detection method, device and equipment provided by the embodiment can obtain a file image of a to-be-identified file, determine a file type of the to-be-identified file based on the file image, perform structured detection on the file image, obtain field attributes of field information in the file image, and then determine structured information included in the to-be-identified file according to the file type and the field attributes, thereby effectively implementing end-to-end structured information detection operation on the to-be-identified file by using global semantic information of the image. The information detection method is not only simple, but also can greatly improve the quality and efficiency of detection of structured information, further improves the practicability of the method, and is conducive to market promotion and application. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 A flowchart of a structured information detection method provided by the embodiment of the present application;

[0038] Figure 2 A principle block diagram of a structured information detection method provided by the embodiment of the present application;

[0039] Figure 3 A flowchart of performing structured detection on the file image to obtain field attributes of field information in the file image provided by the embodiment of the present application;

[0040] Figure 4 A flowchart of performing field detection on the file image based on the category semantic features and image field features to obtain field attributes of field information in the file image provided by the embodiment of the present application;

[0041] Figure 5 A schematic diagram of obtaining field attributes of field information in the file image provided by the embodiment of the present application;

[0042] Figure 6 A flowchart of another structured information detection method provided by the embodiment of the present application;

[0043] Figure 7A flowchart illustrating another method for detecting structured information provided in an embodiment of the present invention;

[0044] Figure 8 This is a schematic diagram of a structured information detection device provided in an embodiment of the present invention;

[0045] Figure 9 To and Figure 8 A schematic diagram of the electronic device corresponding to the structured information detection device provided in the embodiment shown;

[0046] Figure 10 A flowchart illustrating another method for detecting structured information provided in an embodiment of the present invention;

[0047] Figure 11 This is a schematic diagram of another structured information detection device provided in an embodiment of the present invention;

[0048] Figure 12 To and Figure 11 A schematic diagram of the electronic device corresponding to the structured information detection device provided in the embodiment shown;

[0049] Figure 13 A flowchart illustrating another method for detecting structured information provided in an embodiment of the present invention;

[0050] Figure 14 A schematic diagram of the structure of another structured information detection device provided in an embodiment of the present invention;

[0051] Figure 15 To and Figure 14 The illustrated embodiment provides a schematic diagram of the electronic device corresponding to the structured information detection device. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0054] It should be understood that the term "and / or" as used herein merely describes an associated relationship between associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0055] Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as meaning "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0056] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that a product or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such product or system. Without more limitations, the element defined by the phrase "including a" does not exclude the presence of another identical element in the product or system including the element.

[0057] In addition, the step sequence in each of the following method embodiments is only an example, and is not strictly limited.

[0058] In order to facilitate the understanding of the technical solutions of the present application, the prior art is briefly described as follows:

[0059] With the popularization of optical character recognition (OCR) technology, more and more scenarios begin to apply OCR technology to extract the structured information of cards, certificates and bills, in order to help users extract key information such as bank card number, certificate number (used to identify user identity), consumption amount, etc. Different from the extraction of text information of general documents, the structured information extraction of cards, certificates and bills not only extracts the position and content of the text, but also extracts the attributes of the text, for example, the name "Zhang San" on the certificate. Only the position and content of the text block are extracted, and it is not known that "Zhang San" describes what, but if the attribute of the text block is extracted as a name, it can be known that "Zhang San" is the name of the person on the certificate.

[0060] The commonly used structured information extraction method generally comprises the following steps: first, detecting the text of the card, certificate and bill through a text detection model, and then extracting the relationship between the text blocks through a graph convolution network (GCN) combined with the text recognition content, so as to extract the structured information of the card, certificate and bill. However, the above implementation mode has the following disadvantages:

[0061] (1) Non-end-to-end, complex process.

[0062] The above structured information extraction operation utilizes three models of text detection, text recognition and GCN, which will cause low algorithm efficiency and difficulty in optimization. In addition, this method cannot extract structured information end-to-end, and is difficult to be universal, and often can only train a special model for a specific type of card, certificate and bill.

[0063] (2) Global semantic information of the image is not used.

[0064] When the GCN obtains the text block attribute, the text block image information is extracted and combined with the text position information and the text content as the node of the GCN, and is sent into the model to obtain the text block attribute. This process cannot directly send the whole image into the model, so that only the local information is used to infer the text block attribute, and the global semantic of the image is not used, which will reduce the accuracy of extracting the text block attribute.

[0065] (3) Error transmission is prone to occur.

[0066] In the prior art, the text recognition result depends on the accuracy of the text detection, and the GCN obtains the text block attribute depending on the text detection and the text recognition result. Therefore, each link depends on each other, and error transmission is prone to occur, which affects the final information detection result.

[0067] To solve the above technical problems, the embodiment provides a structured information detection method, device and equipment, wherein the method obtains a file image of a to-be-recognized file, the format of the to-be-recognized file is fixed or relatively fixed, then determines the file type of the to-be-recognized file based on the file image, and performs structured detection on the file image to obtain the field attribute of the field information in the file image, and finally determines the structured information included in the to-be-recognized file according to the file type and the field attribute, so that the global semantic information of the image can be used to perform end-to-end structured information detection on the to-be-recognized file, the detection operation is relatively simple, and the quality and efficiency of detecting the structured information can be greatly improved.

[0068] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features in the embodiments can be combined with each other without conflict.

[0069] Figure 1 A flowchart of a structured information detection method provided by an embodiment of the present application is shown in FIG. 1. Figure 2 A schematic diagram of a structured information detection method provided by an embodiment of the present application is shown in FIG. 2. Figures 1-2 As shown in FIG. 1, the embodiment provides a structured information detection method, and an execution subject of the method can be a structured information detection device. It can be understood that the structured information detection device can be implemented as software or a combination of software and hardware. Specifically, the structured information detection method can include the following steps.

[0070] Step S101: Obtain a file image of a to-be-recognized file.

[0071] Step S102: Determine a file type of the to-be-recognized file based on the file image.

[0072] Step S103: Perform structured detection on the file image to obtain a field attribute of field information in the file image.

[0073] Step S104: Determine structured information included in the to-be-recognized file according to the file type and the field attribute.

[0074] The above steps are described in detail as follows.

[0075] Step S101: Obtain a file image of a to-be-recognized file.

[0076] The to-be-recognized file refers to a file that needs to be subjected to a structured information detection operation. In a specific implementation, the number of to-be-recognized files can be one or more. In some instances, the format of the to-be-recognized file is fixed or relatively fixed. It should be noted that the fixed format of the to-be-recognized file can mean that the region positions corresponding to various types of data content in the to-be-recognized file are fixed. The relatively fixed format of the to-be-recognized file can mean that the difference between the region positions corresponding to various types of data content in different to-be-recognized files of the same type is less than or equal to a preset threshold. The embodiment does not limit the specific value of the preset threshold, which can be set according to specific application requirements and design requirements by a person skilled in the art. For example, the difference can be less than or equal to 5%, or less than or equal to 3%, and the like.

[0077] In some examples, the same type of file to be identified corresponds to the same format, and the different types of file to be identified can correspond to the same or different formats. In other examples, the type of file to be identified can include at least one of the following: a bus card, a medical insurance card, a bank card, a driver's license, a certificate, a fixed-format bill, and the like. It can be understood that the type of file to be identified is not limited to the above-mentioned cards, certificates, and bills, and a person skilled in the art can configure the file to be identified according to specific application requirements and design requirements, which will not be described here.

[0078] In addition, the embodiment does not limit the specific implementation of obtaining the file image of the file to be identified, and a person skilled in the art can set it according to specific application requirements and design requirements. For example, the file image of the file to be identified can be stored in a preset area, and the file image of the file to be identified can be obtained by accessing the preset area. Alternatively, after obtaining the file to be identified, the file to be identified can be photographed by a shooting device to obtain the file image of the file to be identified, and then the file image can be actively or passively sent to the structured information detection device by the shooting device, so that the detection device can stably and effectively obtain the file image of the file to be identified.

[0079] Of course, a person skilled in the art can also use other ways to obtain the file image of the file to be identified, as long as the accuracy and reliability of obtaining the file image of the file to be identified can be ensured, which will not be described here.

[0080] Step S102: determining the file type of the file to be identified based on the file image.

[0081] After obtaining the file image, the file image can be analyzed and processed to determine the file type of the file to be identified. The file type of the file to be identified can include at least one of the following: a bus card type, a medical insurance card type, a bank card type, a driver's license type, a certificate type, and a fixed-format bill type.

[0082] In some examples, determining the file type of the file to be identified based on the file image can include: using a machine learning model to analyze and process the file image to determine the file type of the file to be identified, wherein the machine learning model is trained to analyze and process the file image to determine the file type of the file.

[0083] In other examples, determining the file type of the file to be identified based on the file image can include: obtaining an image feature of the card image and a plurality of standard image features for analyzing and processing the card image; determining a target image feature matching the image feature from the plurality of standard image features; and determining the standard card type corresponding to the target image feature as the card type of the card to be identified.

[0084] Of course, the skilled in the art can also determine the file type of the to-be-identified file in other ways as long as the accuracy and reliability of determining the file type of the to-be-identified file can be ensured, which will not be described here.

[0085] Step S103: performing structural detection on the file image to obtain the field attribute of the field information in the file image.

[0086] Among them, for the to-be-identified file, the to-be-identified file can include one or more field information, and each field information can correspond to a field attribute. It can be understood that different field information can correspond to the same or different field attribute. In order to accurately identify the structured information included in the to-be-identified file, after obtaining the file image, the file image can be subjected to structural detection, so as to obtain the field attribute of the field information in the file image.

[0087] In some examples, performing structural detection on the file image to obtain the field attribute of the field information in the file image can include: using a machine learning model to analyze and process the file image to obtain the field attribute of the field information in the file image, wherein the machine learning model is trained to analyze and process the file image to determine the field attribute of the field information in the file.

[0088] For example, when the to-be-identified file is a certificate, after obtaining the file image of the certificate, the file image can be subjected to structural detection, so as to obtain the field attribute of the field information in the file image. For example, the field information included in the file image includes: “Zhang San”, “male”, “Han”, “August 8, 1988”, “No. 2001, Third Street”, “370102198808080088”. After the structural detection of the file image, it can be determined that the field attributes corresponding to the above field information are: “Zhang San”-name, “male”-gender, “Han”-nationality, “August 8, 1988”-date, “No. 2001, Third Street”-address information, and “370102198808080088”-string.

[0089] It should be noted that the execution order of the above steps S103 and S102 in the embodiment is not limited to the order defined by the serial number in the above embodiment. The skilled in the art can set it according to the specific application requirement and design requirement, for example, step S103 can be executed before step S102, or step S103 can be executed simultaneously with step S102, which will not be described here.

[0090] Step S104: determining the structured information included in the to-be-identified file according to the file type and the field attribute.

[0091] After obtaining the file type and the field attribute, the file type and the field attribute can be analyzed to determine the structured information included in the to-be-identified file. The structured information can refer to information that can be decomposed into a plurality of constituent parts associated with each other after analysis, has a clear hierarchical structure, can be managed by a database, and has certain operation specifications.

[0092] In some examples, determining the structured information included in the to-be-identified file according to the file type and the field attribute can include analyzing the file type and the field attribute by using a deep learning model to obtain the structured information included in the to-be-identified file, wherein the deep learning model is trained to determine the structured information included in the file according to the file type and the field attribute.

[0093] For example, when the to-be-identified file is a certificate, after obtaining the file image of the certificate, the file image can be analyzed to determine that the file type of the to-be-identified file is a certificate. In addition, the file image can be subjected to structured detection, so that the field attribute of the field information in the file image can be obtained, such as “Zhang San”-name, “male”-gender, “Han”-nationality, “August 8, 1988”-date, “No. 2001, Third Street”-address information, and “370102198808080088”-string.

[0094] Then, the file type and the field attribute can be analyzed to determine the structured information included in the to-be-identified file. The structured information can be “Zhang San” as the name information on the certificate, “male” as the gender information on the certificate, “Han” as the nationality information on the certificate, “August 8, 1988” as the date of birth on the certificate, “No. 2001, Third Street” as the address on the certificate, and “370102198808080088” as the citizen identification number on the certificate. Thus, accurate detection of the structured information in the to-be-identified file is effectively achieved.

[0095] The structured information detection method provided in this embodiment includes obtaining a file image of a to-be-identified file, determining a file type of the to-be-identified file based on the file image, and performing structured detection on the file image to obtain a field attribute of field information in the file image, and then determining structured information included in the to-be-identified file according to the file type and the field attribute. Thus, end-to-end structured information detection of the to-be-identified file using global semantic information of the image is effectively achieved. The information detection method is not only simple, but also can greatly improve the quality and efficiency of detection of the structured information, further improve the practicality of the method, and is conducive to market promotion and application.

[0096] Figure 3 A flowchart for obtaining the field attribute of the field information in the file image is provided for the embodiment of the present application. Based on the above embodiment, with reference to the accompanying drawings, the embodiment provides an implementation manner for obtaining the field attribute of the field information in the file image. Specifically, the structured detection of the file image and the obtaining of the field attribute of the field information in the file image in the embodiment can include the following steps. Figure 3 The embodiment provides an implementation manner for obtaining the field attribute of the field information in the file image. Specifically, the structured detection of the file image and the obtaining of the field attribute of the field information in the file image in the embodiment can include the following steps.

[0097] Step S301: Obtain the category semantic feature corresponding to the file type and the image field feature corresponding to the file image.

[0098] Step S302: Perform field detection on the file image based on the category semantic feature and the image field feature to obtain the field attribute of the field information in the file image.

[0099] For the to-be-recognized file, the category semantic feature corresponding to the file type is set in advance. It can be understood that different to-be-recognized files of different file types can correspond to different type semantic features, and the same file type can correspond to the same type semantic feature. When performing the structured detection on the file image, the file type can be used as an auxiliary factor to perform the field detection.

[0100] Specifically, after obtaining the file type, the file type can be analyzed and processed to obtain the category semantic feature corresponding to the file type. After obtaining the file image, the file image can be analyzed and recognized to obtain the image field feature corresponding to the file image. After obtaining the image field feature and the category semantic feature, the file image can be detected based on the category semantic feature and the image field feature to obtain the field attribute of the field information in the file image.

[0101] In some examples, for the to-be-recognized file, the number of the recognized field information is greater than or equal to the number of the field attribute. For example, the address information of the certificate can be composed of two field information, one of which can be the information located in the first line, and the other of which can be the information located in the second line. Since the two field information can identify the complete address information, the two field information correspond to the same field attribute “address attribute”. At this time, the number of the field information is greater than the number of the field attribute.

[0102] In the embodiment, the category semantic features corresponding to the file type and the image field features corresponding to the file image are acquired, and then the field detection is performed on the file image based on the category semantic features and the image field features to obtain the field attribute of the field information in the file image, so that the accuracy of analyzing and identifying the field attribute of the field information is effectively ensured, and the accuracy of detecting the structured information of the to-be-identified file is improved.

[0103] Figure 4 The flowchart of obtaining the field attribute of the field information in the file image based on the category semantic features and the image field features is provided in the embodiment of the application. Figure 5 The schematic diagram of obtaining the field attribute of the field information in the file image is provided in the embodiment of the application. In continuation of the above embodiment, with reference to FIG. 6, the specific implementation manner of obtaining the field attribute of the field information in the file image is not limited in the embodiment, and the person skilled in the art can set it according to the specific application requirement and design requirement. More preferably, the field detection on the file image based on the category semantic features and the image field features to obtain the field attribute of the field information in the file image in the embodiment can include: Figures 4-5

[0104] Step S401: performing normalization processing on the image field features and the category semantic features to obtain a first weight matrix corresponding to the image field features and a second weight matrix corresponding to the category semantic features.

[0105] Step S402: obtaining the field attribute of the field information in the file image according to the image field features, the category semantic features, the first weight matrix and the second weight matrix.

[0106] After the image field features and the category semantic features are acquired, the image field features and the category semantic features can be normalized respectively, so that the first weight matrix corresponding to the image field features and the second weight matrix corresponding to the category semantic features can be obtained.

[0107] After the image field features, the category semantic features, the first weight matrix and the second weight matrix are acquired, the image field features, the category semantic features, the first weight matrix and the second weight matrix can be analyzed and processed to obtain the field attribute of the field information in the file image. In some instances, obtaining the field attribute of the field information in the file image according to the image field features, the category semantic features, the first weight matrix and the second weight matrix can include:

[0108] Step S4021: determining a first reference feature based on the first weight matrix and the category semantic features.

[0109] ​The determining the first reference feature based on the first weight matrix and the category semantic feature can include: extracting, from the category semantic feature, a category reference feature corresponding to the first weight matrix; and determining the first reference feature based on the first weight matrix and the category reference feature.

[0110] Specifically, the network structure for implementing the detection operation of the structured information can be set to include an image classification network branch and a structured detection branch, where the image classification network branch is used to determine the file type of the file image, and the structured detection branch is used to determine the field attribute of the field information in the file image. When the field detection operation is performed in the structured detection branch, the image classification network branch can actively extract, from the category semantic feature, a category reference feature corresponding to the first weight matrix. In specific implementation, the category reference feature corresponding to the first weight matrix can be extracted based on application requirements and configuration requirements. It can be understood that the category reference feature can be at least part of the category semantic feature.

[0111] After the category reference feature is obtained, the first weight matrix and the category reference feature can be analyzed and processed to determine the first reference feature. In some examples, the determining the first reference feature based on the first weight matrix and the category reference feature can include: determining the product of the first weight matrix and the category reference feature as the first reference feature, thereby effectively ensuring the accuracy and reliability of the determination of the first reference feature.

[0112] Step S4022: determining a second reference feature based on a second weight matrix and an image field feature.

[0113] After the second weight matrix and the image field feature are obtained, the second weight matrix and the image field feature can be analyzed and processed to determine the second reference feature. In some examples, the determining the second reference feature based on the second weight matrix and the image field feature can include: determining the product of the second weight matrix and the image field feature as the second reference feature, thereby effectively ensuring the accuracy and reliability of the determination of the second reference feature.

[0114] Step S4023: obtaining the field attribute of the field information in the file image according to the first reference feature, the second reference feature, and the image field feature.

[0115] After the first reference feature, the second reference feature and the image field feature are acquired, the first reference feature, the second reference feature and the image field feature can be analyzed and processed to obtain the field attribute of the field information in the file image. In some examples, obtaining the field attribute of the field information in the file image according to the first reference feature, the second reference feature and the image field feature can include: performing weighted summation on the first reference feature, the second reference feature and the image field feature to obtain the field attribute of the field information in the file image, thereby realizing accurate and effective identification of the field attribute of the field information in the file image, and further improving the accuracy and reliability of detecting structured information.

[0116] Figure 6 Another flowchart of a structured information detection method provided by an embodiment of the present application is shown in FIG. 6. Based on the above embodiment, the method of the present embodiment can further include the following steps. Figure 6

[0117] Step S601: Acquire a preset card image used to configure the number of channels of the last layer feature map in the deep learning model.

[0118] Step S602: Configure the number of channels of the last layer feature map according to the preset card image.

[0119] In the process of training and generating the deep learning model, in order to improve the practicability of the deep learning model, the related parameters of the deep learning model can be configured, for example, the number of channels of the last layer feature map in the deep learning model can be configured. It can be understood that when the number of channels of the last layer feature map in the deep learning model is set to a higher parameter, the deep learning model can perform structured information detection operation on most types of card images, and at the same time, the speed of the deep learning model performing structured information detection operation will be reduced, and the efficiency will be reduced. When the number of channels of the last layer feature map in the deep learning model is set to a lower parameter, the deep learning model can perform structured information detection operation on a smaller number of types of card images, and at the same time, the speed of the deep learning model performing structured information detection operation will be improved, and the efficiency will be improved.

[0120] ​In order to be able to configure the channel number of the last layer feature map in the deep learning model, a preset card certificate image corresponding to a preset type of card certificate can be obtained. The preset card certificate image includes a number of field attributes of field information, which is the largest compared to the number of field attributes of field information included in the existing card certificate image that can be obtained. Therefore, obtaining the preset card certificate image for configuring the channel number of the last layer feature map in the deep learning model can include: obtaining all alternative card certificate images for configuring the channel number of the last layer feature map in the deep learning model, determining the number of field attributes corresponding to each alternative card certificate image, and determining the alternative card certificate image with the largest number of field attributes as the preset card certificate image for configuring the channel number of the last layer feature map in the deep learning model.

[0121] Specifically, in order to ensure the practicability and scope of application of the deep learning model, all alternative card certificate images capable of detecting structured information can be obtained. Different alternative card certificate images can include different field information, and different field information can correspond to the same or different field attributes. It can be understood that different alternative card certificate images can correspond to different numbers of field attributes. Then the number of field attributes corresponding to each alternative card certificate image can be determined, and the alternative card certificate image with the largest number of field attributes can be determined as the preset card certificate image for configuring the channel number of the last layer feature map in the deep learning model, thereby effectively ensuring the accuracy and reliability of determining the preset card certificate image.

[0122] After obtaining the preset card certificate image, the channel number of the last layer feature map can be configured according to the preset card certificate image. In some examples, configuring the channel number of the last layer feature map according to the preset card certificate image can include: obtaining a preset field attribute of a preset field information in the preset card certificate image; counting the number of preset field attributes; and configuring the channel number of the last layer feature map based on the number of preset field attributes, wherein the channel number of the last layer feature map is greater than the number of preset field attributes.

[0123] Specifically, after obtaining the preset card certificate image, the preset card certificate image can be analyzed and processed to obtain a preset field attribute of a preset field information in the preset card certificate image. Then the number of preset field attributes can be counted, and the channel number of the last layer feature map can be configured based on the number of preset field attributes. It should be noted that the channel number of the last layer feature map after configuration is greater than the number of preset field attributes, which can enable the generated deep learning model to perform structured information detection operation on all obtainable card certificate images, further improving the practicability and scope of application of the method.

[0124] In some examples, the last layer feature map includes multiple channels, each channel corresponding to a field attribute, and different channels corresponding to different field attributes.

[0125] To improve the flexibility and reliability of the method, after the number of channels of the last layer feature map is configured, the last layer feature map can include multiple channels, for different types of file images of the to-be-identified files, for the same channel, different file images correspond to the same or different field attributes. Figure 1 Generally, the last layer feature map can include multiple channels, for a file image, each channel can correspond to a field attribute, and different channels correspond to different field attributes.

[0126] For example, a file image of a to-be-identified file can include field 1, field 2, field 3, and field 4, wherein field 1 corresponds to field attribute 1, field 2 corresponds to field attribute 2, field 3 corresponds to field attribute 3, and field 4 corresponds to field attribute 4. At this time, in the detection operation of the structured information of the above file image, channel 1 in the last layer feature map can correspond to field attribute 1, channel 2 can correspond to field attribute 2, channel 3 can correspond to field attribute 3, and channel 4 can correspond to field attribute 4, so as to facilitate the analysis and identification operation of different field attributes of the same file based on the deep learning model, and further improve the accuracy and reliability of the detection of structured information.

[0127] In other examples, the last layer feature map includes multiple channels, for the same channel, different file images of different file types correspond to the same or different field attributes.

[0128] To improve the flexibility and reliability of the method, after the number of channels of the last layer feature map is configured, the last layer feature map can include multiple channels, for different types of file images of the to-be-identified files, for the same channel, different file images correspond to the same or different field attributes.

[0129] For example, a file image A of a to-be-identified file A can include field 1, field 2, field 3, and field 4, field 1 corresponds to field attribute a1, field 2 corresponds to field attribute a2, field 3 corresponds to field attribute a3, and field 4 corresponds to field attribute a4; a file image B of a to-be-identified file B can include field 1, field 2, and field 3, field 1 corresponds to field attribute b1, field 2 corresponds to field attribute b2, and field 3 corresponds to field attribute b3.

[0130] For the above two file images (file image A and file image B), channel 1 in the last layer feature map can correspond to field attribute a1 and field attribute b1, channel 2 corresponds to field attribute a2 and field attribute b2, channel 3 corresponds to field attribute a3 and field attribute b3, and channel 4 corresponds to field attribute a4, so as to facilitate the simultaneous recognition of different file types of the to-be-recognized file based on the deep learning model. The structural information recognition operation is further improved. The flexibility and reliability of detecting structural information are further improved.

[0131] Figure 7 The flowchart of another structural information detection method provided by the embodiment of the application is shown in the figure. Based on the above embodiment, the method in the embodiment can further include: Figure 7

[0132] Step S701: Obtain the reference field attribute of the reference field information in the reference card.

[0133] Step S702: Count the number of reference field attributes.

[0134] Step S703: When the number of reference field attributes is greater than the number of channels of the last layer feature map, update the number of channels of the last layer feature map based on the number of reference field attributes, wherein the number of channels of the updated last layer feature map is greater than the number of reference field attributes.

[0135] Step S704: Generate an updated deep learning model according to the updated last layer feature map.

[0136] For the to-be-recognized file, the file type of the to-be-recognized file can increase with the increase of the use scene and application demand, wherein the number of field attributes of the field information in the to-be-recognized file of the new file type can be greater than, less than or equal to the number of field attributes of the field information in the to-be-recognized file of the original file type. When the number of field attributes of the field information in the to-be-recognized file of the new file type is greater than the number of field attributes of the field information in the to-be-recognized file of the original file type, the pre-trained deep learning model can not be able to detect the structural information of the to-be-recognized file of the new file type. At this time, the deep learning model can be updated based on the to-be-recognized file of the new file type.

[0137] Therefore, in order to ensure the accuracy and reliability of detecting structural information, when the reference card that needs to be detected for structural information is obtained, the reference field attribute of the reference field information in the reference card can be obtained. The specific implementation of the reference field attribute is similar to the specific implementation of the field attribute in the above embodiment. For specific implementation, please refer to the above statements. Here, it is not repeated. ​

[0138] After the reference field attribute of the reference field information in the reference card is acquired, the number of the reference field attribute is counted, and then the number of the reference field attribute is compared with the channel number of the last layer feature map. When the number of the reference field attribute is less than or equal to the channel number of the last layer feature map, it is indicated that the pre-trained deep learning model can perform the detection operation of the structured information of the reference card. At this time, the deep learning model can be directly used to perform the detection operation of the structured information of the reference card, so that the structured information corresponding to the reference card can be accurately and effectively acquired.

[0139] When the number of the reference field attribute is greater than the channel number of the last layer feature map, it is indicated that the pre-trained deep learning model cannot perform the detection operation of the structured information of the reference card. At this time, in order to improve the practicability and application range of the deep learning model, the channel number of the last layer feature map can be updated based on the number of the reference field attribute. It should be noted that the updated channel number of the last layer feature map is greater than the number of the reference field attribute. After the updated last layer feature map is generated, the updated deep learning model can be generated according to the updated last layer feature map. The updated deep learning model can perform the detection operation of the structured information of the reference card, further improving the practicability and application range of the method.

[0140] In this embodiment, the reference field attribute of the reference field information in the reference card is acquired, and then the number of the reference field attribute is counted. When the number of the reference field attribute is greater than the channel number of the last layer feature map, the channel number of the last layer feature map is updated based on the number of the reference field attribute, and the updated deep learning model is generated according to the updated last layer feature map. In this way, when the deep learning model cannot accurately perform the detection operation of the structured information of the reference card, the deep learning model can be updated based on the reference card in time, so that the updated deep learning model can perform the detection operation of the structured information of the reference card, further improving the practicability and application range of the method.

[0141] In specific applications, the application embodiment provides an end-to-end structured information detection method. The method can detect the structured information of various types of to-be-recognized files through shared channels, so that the method has universality, thereby improving the practicability and application range of the method. In addition, the execution subject of the detection method can be a structured information detection network, which can include a structure detection branch and a network classification branch. Specifically, when the detection network executes the above detection method, the following steps can be included:

[0142] Step 1: Obtain an image to be recognized of a card, certificate or bill to be recognized.

[0143] Step 2: input the image to be recognized into a detection network to obtain a card, certificate or bill category corresponding to the image to be recognized and field attributes of field information in the image to be recognized.

[0144] Specifically, after obtaining the image to be recognized, the image to be recognized can be analyzed and processed by using a network classification branch to obtain a card, certificate or bill category corresponding to the image to be recognized. At the same time, the image to be recognized can be structurally detected by using a structure detection branch to obtain field attributes of field information in the image to be recognized.

[0145] Step 3: determine structured information included in the card, certificate or bill to be recognized according to the card, certificate or bill category and the field attributes.

[0146] Specifically, the card, certificate or bill category semantic information learned by the image classification branch can be transmitted to the structure detection branch of the card, certificate or bill by using an attention mechanism Attention, so that the structure detection branch can learn the category semantic information of the card, certificate or bill, and then determine the structured information included in the card, certificate or bill to be recognized based on the category semantic information and the field attributes, effectively improving the accuracy and reliability of identifying the structured information.

[0147] In addition, the detection network capable of realizing the structured information detection operation in the application embodiment can realize a shared channel. Specifically, the same channel of the last feature map in the detection network can correspond to field attributes corresponding to different types of card, certificate or bill, which is conducive to realizing the structured information detection operation of multiple card, certificate or bill without changing the structure of the detection network, but only changing the number of channels of the last feature map.

[0148] For example, the first field attributes of the bill category 1, the bill category 2, …, the bill category n can share the first channel, the second field attributes of the bill category 1, the bill category 2, …, the bill category n can share the second channel, and so on.

[0149] The end-to-end structured information detection method provided by the application embodiment can realize end-to-end structured information detection operation by using a simple CNN network. The detection operation is simple, and the detection efficiency is greatly improved. In addition, the method can realize end-to-end structured information detection operation. The input model is the full image of the card certificate, and therefore, the global semantic information of the image is effectively used for the structured information detection operation. In this way, when the image quality is not high, scanning is offset, the text is blurred, and the like, the field information corresponding to the field attribute in the image can still be accurately recognized due to the global semantic information of the image. In addition, the structured information detection process is a forward inference process of the CNN network, and there is no error propagation, thereby effectively improving the accuracy and reliability of the detection of the structured information of the card certificate, further ensuring the practicability of the method, and being conducive to the market promotion and application.

[0150] Figure 8 A structural diagram of a structured information detection device provided by the application embodiment is shown in FIG. 1. As shown in FIG. 1, the application embodiment provides a structured information detection device. The detection device can perform the structured information detection method shown in FIG. 2. Figure 8 The detection device can include a first acquisition module 11, a first determination module 12, a first detection module 13, and a first processing module 14. Figure 1 The detection device can include a first acquisition module 11, a first determination module 12, a first detection module 13, and a first processing module 14.

[0151] The first acquisition module 11 is configured to acquire a file image of a to-be-recognized file.

[0152] The first determination module 12 is configured to determine a file type of the to-be-recognized file based on the file image.

[0153] The first detection module 13 is configured to perform structured detection on the file image to obtain a field attribute of field information in the file image.

[0154] The first processing module 14 is configured to determine structured information included in the to-be-recognized file according to the file type and the field attribute.

[0155] In some examples, the layout of the to-be-recognized file is fixed or relatively fixed.

[0156] In some examples, the to-be-recognized file can include at least one of the following: a bus card, a medical insurance card, a bank card, a driver's license, a certificate, and a fixed-layout bill.

[0157] In some examples, when the first detection module 13 performs structural detection on the file image to obtain the field attribute of the field information in the file image, the first detection module 13 can be configured to perform: obtaining a category semantic feature corresponding to the file type and an image field feature corresponding to the file image; performing field detection on the file image based on the category semantic feature and the image field feature to obtain the field attribute of the field information in the file image.

[0158] In some examples, the number of field information is greater than or equal to the number of field attributes.

[0159] In some examples, when the first detection module 13 performs field detection on the file image based on the category semantic feature and the image field feature to obtain the field attribute of the field information in the file image, the first detection module 13 can be configured to perform: performing normalization processing on the image field feature and the category semantic feature to obtain a first weight matrix corresponding to the image field feature and a second weight matrix corresponding to the category semantic feature; and obtaining the field attribute of the field information in the file image according to the image field feature, the category semantic feature, the first weight matrix, and the second weight matrix.

[0160] In some examples, when the first detection module 13 obtains the field attribute of the field information in the file image according to the image field feature, the category semantic feature, the first weight matrix, and the second weight matrix, the first detection module 13 can be configured to perform: determining a first reference feature based on the first weight matrix and the category semantic feature; determining a second reference feature based on the second weight matrix and the image field feature; and obtaining the field attribute of the field information in the file image according to the first reference feature, the second reference feature, and the image field feature.

[0161] In some examples, when the first detection module 13 determines the first reference feature based on the first weight matrix and the category semantic feature, the first detection module 13 can be configured to perform: extracting a category reference feature corresponding to the first weight matrix from the category semantic feature; and determining the first reference feature based on the first weight matrix and the category reference feature.

[0162] In some examples, when the first detection module 13 determines the first reference feature based on the first weight matrix and the category reference feature, the first detection module 13 can be configured to perform: determining the product of the first weight matrix and the category reference feature as the first reference feature.

[0163] In some examples, when the first detection module 13 determines the second reference feature based on the second weight matrix and the image field feature, the first detection module 13 can be configured to perform: determining the product of the second weight matrix and the image field feature as the second reference feature.

[0164] In some examples, when the first detection module 13 obtains the field attribute of the field information in the file image according to the first reference feature, the second reference feature and the image field feature, the first detection module 13 can be configured to perform: performing weighted summation on the first reference feature, the second reference feature and the image field feature to obtain the field attribute of the field information in the file image.

[0165] In some examples, when the first processing module 14 determines the structured information included in the to-be-recognized file according to the file type and the field attribute, the first processing module 14 can be configured to perform: performing analysis and processing on the file type and the field attribute by using a deep learning model to obtain the structured information included in the to-be-recognized file, wherein the deep learning model is trained to determine the structured information included in the file according to the file type and the field attribute.

[0166] In some examples, the first acquisition module 11 and the first processing module 14 in the embodiment can be configured to perform the following steps:

[0167] The first acquisition module 11 is configured to acquire a preset card image used for configuring a number of channels of a last layer feature map in a deep learning model.

[0168] The first processing module 14 is configured to configure the number of channels of the last layer feature map according to the preset card image.

[0169] In some examples, when the first processing module 14 configures the number of channels of the last layer feature map according to the preset card image, the first processing module 14 can be configured to perform: acquiring a preset field attribute of preset field information in the preset card image; counting a number of the preset field attributes; and configuring the number of channels of the last layer feature map based on the number of the preset field attributes, wherein the number of channels of the last layer feature map is greater than the number of the preset field attributes.

[0170] In some examples, the last layer feature map includes a plurality of channels, each channel corresponds to a field attribute, and different channels correspond to different field attributes.

[0171] In some examples, the last layer feature map includes a plurality of channels, and for the same channel, the file images of different file types correspond to the same or different field attributes.

[0172] In some examples, the first acquisition module 11 and the first processing module 14 in the embodiment can be configured to perform the following steps:

[0173] The first acquisition module 11 is configured to acquire a reference field attribute of reference field information in a reference card.

[0174] The first processing module 14 is configured to count the number of reference field attributes, and when the number of reference field attributes is greater than the number of channels of the last layer feature map, update the number of channels of the last layer feature map based on the number of reference field attributes, wherein the updated number of channels of the last layer feature map is greater than the number of reference field attributes; and generate an updated deep learning model according to the updated last layer feature map.

[0175] Figure 8 The apparatus can perform the method of the embodiments Figures 1-7 The method of the embodiments is not described in detail, and reference can be made to the related description of the Figures 1-7 The embodiments. The execution process and technical effects of the technical solution are described in the Figures 1-7 The embodiments, which will not be repeated here.

[0176] In one possible design, Figure 8 The structured information detection apparatus can be implemented as an electronic device, which can be a mobile phone, a tablet computer, a server, or various devices. As Figure 9 The electronic device can include a first processor 21 and a first memory 22. The first memory 22 is configured to store programs of the structured information detection method provided in the Figures 1-7 The embodiments, and the first processor 21 is configured to execute the programs stored in the first memory 22.

[0177] The programs include one or more computer instructions, and the one or more computer instructions can implement the following steps when executed by the first processor 21:

[0178] Obtain a file image of a file to be recognized;

[0179] Determine a file type of the file to be recognized based on the file image;

[0180] Perform structured detection on the file image to obtain field attributes of field information in the file image;

[0181] Determine structured information included in the file to be recognized according to the file type and the field attributes.

[0182] Further, the first processor 21 is further configured to execute all or part of the steps in the foregoing Figures 1-7 The embodiments.

[0183] The structure of the electronic device can further include a first communication interface 23, which is configured to communicate with other devices or communication networks.

[0184] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for an electronic device, which comprises computer software instructions for executing the above Figures 1-7 The method for detecting structured information in the method embodiment shown.

[0185] Figure 10 Another flowchart of a method for detecting structured information is provided in the embodiment of the present application. Figure 10 As shown in the method embodiment, the embodiment provides another method for detecting structured information, and the execution subject of the method can be a structured information detection device. It can be understood that the structured information detection device can be implemented as software or a combination of software and hardware. Specifically, the method for detecting structured information can comprise the following steps.

[0186] S1001: In response to a request for calling an information recognition service, determining a processing resource corresponding to the information recognition service.

[0187] S1002: Using the processing resource to perform the following steps: obtaining a file image of a to-be-recognized file; determining a file type of the to-be-recognized file based on the file image; performing structured detection on the file image to obtain field attributes of field information in the file image; and determining structured information included in the to-be-recognized file according to the file type and the field attributes.

[0188] Specifically, the method for detecting structured information provided by the present application can be executed in the cloud. A plurality of computing nodes can be deployed in the cloud, and each computing node has computing, storage and other processing resources. In the cloud, a plurality of computing nodes can be organized to provide a certain service. Of course, one computing node can also provide one or more services.

[0189] For the scheme provided by the present application, the cloud can provide a service for completing the method for detecting structured information, which is called a structured information detection service. When a user needs to use the structured information detection service, the structured information detection service is called to trigger a request for calling the structured information detection service to the cloud. The file image obtained by shooting the to-be-recognized file can be carried in the request. The cloud determines the computing node responding to the request, and uses the processing resource in the computing node to perform the following steps: obtaining a file image of a to-be-recognized file; determining a file type of the to-be-recognized file based on the file image; performing structured detection on the file image to obtain field attributes of field information in the file image; and determining structured information included in the to-be-recognized file according to the file type and the field attributes.

[0190] Specifically, the implementation process, implementation principle and implementation effect of the above method steps in the embodiment are the same as those of the aboveFigures 1-7 The implementation process, implementation principle and implementation effect of the method steps of the embodiment are similar, and the parts not described in detail in the embodiment can refer to the related description of the embodiment. Figures 1-7 The embodiment provides another structured information detection device.

[0191] Figure 11 Another structural diagram of the structured information detection device provided by the embodiment of the application is shown in the figure. Figure 11 The embodiment provides another structured information detection device, which can perform the structured information detection method described above. Figure 10 The structured information detection device can include a second determination module 31 and a second processing module 32.

[0192] The second determination module 31 is configured to determine a processing resource corresponding to the information identification service in response to a request for calling the information identification service.

[0193] The second processing module 32 is configured to perform the following steps by using the processing resource: obtaining a file image of a to-be-identified file; determining a file type of the to-be-identified file based on the file image; performing structured detection on the file image to obtain a field attribute of field information in the file image; and determining structured information included in the to-be-identified file according to the file type and the field attribute.

[0194] Figure 11 The structured information detection device can perform the method of the embodiment. Figure 10 The parts not described in detail in the embodiment can refer to the related description of the embodiment. Figure 10 The execution process and technical effects of the technical solution can refer to the description in the embodiment, which will not be repeated here. Figure 10

[0195] In one possible design, Figure 11 The structure of the structured information detection device can be implemented as an electronic device, which can be various devices such as a mobile phone, a tablet computer, a server, etc. Figure 12 As shown in the figure, the electronic device can include a second processor 41 and a second memory 42. Figure 10 The second memory 42 is configured to store programs of the structured information detection method provided in the embodiment.

[0196] The programs include one or more computer instructions, and the one or more computer instructions can implement the following steps when executed by the second processor 41.

[0197] ​In response to the request for invoking the information identification service, a processing resource corresponding to the information identification service is determined.

[0198] With the processing resource, the following steps are performed: obtaining a file image of a file to be identified; determining a file type of the file to be identified based on the file image; performing structural detection on the file image to obtain field attributes of field information in the file image; and determining structured information included in the file to be identified according to the file type and the field attributes.

[0199] Further, the second processor 41 is further configured to perform all or part of the steps in the embodiments shown in the foregoing Figure 10

[0200] The electronic device can further include a second communication interface 43 for communication between the electronic device and other devices or communication networks.

[0201] In addition, the embodiments of the present application provide a computer storage medium for storing computer software instructions for the electronic device, which includes programs for performing the detection method of structured information in the embodiments of the method shown in the foregoing Figure 10

[0202] Figure 13 The flowchart of another detection method of structured information provided by the embodiments of the present application is shown in the foregoing Figure 13 The embodiments provide another detection method of structured information, and the execution subject of the method can be a detection device of structured information. It can be understood that the detection device of structured information can be implemented as software or a combination of software and hardware. Specifically, the detection method of structured information can include:

[0203] Step S1301: obtaining an information identification request for an image to be identified.

[0204] Step S1302: determining structured information included in the image to be identified based on the information identification request, the structured information being determined based on a file type of the image to be identified and field attributes of field information in the image to be identified, the field attributes of field information in the image to be identified being obtained by performing structural detection on the image to be identified.

[0205] Step S1303: displaying the structured information included in the image to be identified.

[0206] ​​Specifically, the method for detecting structured information can be executed on a client, and the client can be deployed with a plurality of computing nodes, each of which has computing, storage and other processing resources. On the client, a plurality of computing nodes can be organized to provide a certain service, and of course, one computing node can also provide one or more services.

[0207] For the scheme provided by the present application, the client can be provided with a service for completing the method for detecting structured information, referred to as an application service, for example: the detection operation of structured information can be realized through an application app installed on the client. When the user needs to use the structured information detection service, the above-mentioned application app is called, and the user can upload the image to be recognized through the above-mentioned app, so as to obtain an information recognition request for the image to be recognized. The app can determine the computing node responding to the image to be recognized, and use the processing resources in the computing node to execute the following steps: determining the structured information included in the image to be recognized based on the information recognition request, the structured information being determined based on the file type of the image to be recognized and the field attribute of the field information in the image to be recognized, the field attribute of the field information in the image to be recognized being obtained by performing structured detection on the image to be recognized.

[0208] After obtaining the structured information, in order to enable the user to know the structured information in time, the structured information included in the image to be recognized can be displayed through a display interface, thereby effectively improving the practicability of the method.

[0209] Specifically, the implementation process, implementation principle and implementation effect of the above method steps in the embodiment are similar to those of the method steps in the method for detecting structured information provided in the above-mentioned Figures 1-7 embodiment, and the parts not described in detail in the embodiment can refer to the related description of the above-mentioned Figures 1-7 embodiment.

[0210] Figure 14 Another structural diagram of a structured information detection device provided by the embodiment of the present application is shown in the accompanying drawings. Figure 14 The embodiment provides another structured information detection device, which can execute the method for detecting structured information provided in the above-mentioned Figure 13 The detection device can include a third acquisition module 51, a third determination module 52 and a third display module 53. Specifically,

[0211] The third acquisition module 51 is configured to acquire an information recognition request for an image to be recognized.

[0212] The third determining module 52 is configured to determine the structured information included in the image to be recognized based on the information recognition request, the structured information being determined based on a file type of the image to be recognized and a field attribute of field information in the image to be recognized, the field attribute of the field information in the image to be recognized being obtained by performing the structured detection on the image to be recognized.

[0213] The third display module 53 is configured to display the structured information included in the image to be recognized.

[0214] Figure 14 The apparatus can perform the method of the embodiments Figure 13 The method of the embodiments is not described in detail, and reference can be made to the related description of the Figure 13 The embodiments. The execution process and technical effects of the technical solution are described in the embodiments, which will not be described here. Figure 13

[0215] In one possible design, Figure 14 The structured information detection apparatus can be implemented as an electronic device, which can be various devices such as a mobile phone, a tablet computer, and a server. As Figure 15 The electronic device can include a third processor 61 and a third memory 62. The third memory 62 is configured to store a program of the structured information detection method provided in the embodiments, and the third processor 61 is configured to execute the program stored in the third memory 62. Figure 13 The program includes one or more computer instructions, and the one or more computer instructions can implement the following steps when executed by the third processor 61:

[0216] Obtaining an information recognition request for an image to be recognized;

[0217] Determining structured information included in the image to be recognized based on the information recognition request, the structured information being determined based on a file type of the image to be recognized and a field attribute of field information in the image to be recognized, the field attribute of the field information in the image to be recognized being obtained by performing the structured detection on the image to be recognized;

[0218] Displaying the structured information included in the image to be recognized.

[0219] Further, the third processor 61 is further configured to perform all or part of the steps in the foregoing

[0220] Embodiments. Figure 13

[0221] ​​Among them, the structure of the electronic device can further include a third communication interface 63 for communication between the electronic device and other devices or communication networks.

[0222] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for the electronic device, which includes computer software instructions for executing the above-mentioned Figure 13 The program involved in the detection method of the structured information in the method embodiment shown.

[0223] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0224] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of a general hardware platform as necessary, and of course can also be realized by means of combination of hardware and software. Based on such understanding, the above technical solutions can be embodied in the form of computer products, and the present application can adopt the form of computer program products implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0225] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable devices to produce a machine, so that the instructions executed by the computer or other programmable devices produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 the device that implements the function specified in one or more flows or blocks.

[0226] These computer program instructions can also be stored in a computer readable storage medium capable of guiding the computer or other programmable devices to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices, which implement the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocksFigure 1 the function specified in one or more blocks.

[0227] These computer program instructions can also be loaded into computer or other programmable devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer-implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 the flowchart block or blocks. Figure 1 the function specified in one or more blocks.

[0228] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0229] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about the operating environment. This memory is an example of computer readable media. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), programmable gate array (PGA), or flash memory. While the memory is computer readable media, it is not transitory.

[0230] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory computer readable media, such as modulated data signals and carrier waves.

[0231] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of detecting structured information, characterized by, The method comprises the following steps: acquiring a file image of a file to be identified; determining a file type of the file to be identified based on the file image; acquiring category semantic features corresponding to the file type and image field features corresponding to the file image; performing field detection on the file image based on the category semantic features and the image field features to obtain field attributes of field information in the file image; determining structured information included in the file to be identified according to the file type and the field attributes.

2. The method of claim 1, wherein, The layout of the file to be identified is fixed or relatively fixed.

3. The method of claim 1, wherein, The number of the field information is greater than or equal to the number of the field attributes.

4. The method of claim 1, wherein, The field detection on the file image based on the category semantic features and the image field features to obtain the field attributes of the field information in the file image comprises the following steps: performing normalization processing on the image field features and the category semantic features to obtain a first weight matrix corresponding to the image field features and a second weight matrix corresponding to the category semantic features; obtaining the field attributes of the field information in the file image according to the image field features, the category semantic features, the first weight matrix and the second weight matrix.

5. The method of claim 4, wherein, The obtaining of the field attributes of the field information in the file image according to the image field features, the category semantic features, the first weight matrix and the second weight matrix comprises the following steps: determining first reference features based on the first weight matrix and the category semantic features; determining second reference features based on the second weight matrix and the image field features; obtaining the field attributes of the field information in the file image according to the first reference features, the second reference features and the image field features.

6. The method of claim 5, wherein, The determining of the first reference features based on the first weight matrix and the category semantic features comprises the following steps: extracting category reference features corresponding to the first weight matrix from the category semantic features; determining the first reference features based on the first weight matrix and the category reference features.

7. The method of claim 6, wherein, The determining of the first reference features based on the first weight matrix and the category reference features comprises the following steps: determining the product of the first weight matrix and the category reference features as the first reference features.

8. The method of claim 5, wherein, The determining of the second reference features based on the second weight matrix and the image field features comprises the following steps: determining the product of the second weight matrix and the image field features as the second reference features.

9. The method of claim 5, wherein, The obtaining of the field attributes of the field information in the file image according to the first reference features, the second reference features and the image field features comprises the following steps: performing weighted summation on the first reference features, the second reference features and the image field features to obtain the field attributes of the field information in the file image.

10. The method according to any one of claims 3-9, characterized in that, The determining of the structured information included in the file to be identified according to the file type and the field attributes comprises the following steps: The file type and the field attribute are analyzed and processed by using a deep learning model to obtain structured information included in the to-be-identified file, wherein the deep learning model is trained to determine structured information included in a file according to a file type and a field attribute.

11. The method of claim 10, wherein, The method further comprises: obtaining a preset card image used for configuring a channel number of a last layer feature map in the deep learning model; configuring the channel number of the last layer feature map according to the preset card image.

12. The method of claim 11, wherein, The configuration of the channel number of the last layer feature map according to the preset card image comprises: obtaining a preset field attribute of preset field information in the preset card image; counting the number of the preset field attribute; configuring the channel number of the last layer feature map based on the number of the preset field attribute, wherein the channel number of the last layer feature map is greater than the number of the preset field attribute.

13. The method of claim 11, wherein, The last layer feature map comprises a plurality of channels, each channel corresponding to a field attribute, and different channels corresponding to different field attributes.

14. The method of claim 11, wherein, The last layer feature map comprises a plurality of channels, and for the same channel, file images of different file types correspond to the same or different field attributes.

15. The method of claim 11, wherein, The method further comprises: obtaining a reference field attribute of reference field information in a reference card; counting the number of the reference field attribute; when the number of the reference field attribute is greater than the channel number of the last layer feature map, updating the channel number of the last layer feature map based on the number of the reference field attribute, wherein the updated channel number of the last layer feature map is greater than the number of the reference field attribute; generating an updated deep learning model according to the updated last layer feature map.

16. The method of any of claims 1-9, wherein, The to-be-identified file comprises at least one of the following: a bus card, a medical insurance card, a bank card, a driver's license, a certificate, and a fixed format bill.

17. A structured information detection apparatus, characterized by comprising: comprises: a first obtaining module configured to obtain a file image of a to-be-identified file, wherein the to-be-identified file has a fixed or relatively fixed format; a first determining module configured to determine a file type of the to-be-identified file based on the file image; a first detecting module configured to obtain a category semantic feature corresponding to the file type and an image field feature corresponding to the file image; performing field detection on the file image based on the category semantic feature and the image field feature to obtain a field attribute of field information in the file image; a first processing module configured to determine structured information included in the to-be-identified file according to the file type and the field attribute.

18. An electronic device, comprising: comprises: a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the structured information detection method in any one of claims 1-16.

19. A method of detecting structured information, characterized by, comprises: in response to a request for invoking an information identification service, determining a processing resource corresponding to the information identification service; The processing resource is used to perform the following steps: obtaining a file image of a file to be identified; determining a file type of the file to be identified based on the file image; obtaining category semantic features corresponding to the file type and image field features corresponding to the file image; performing field detection on the file image based on the category semantic features and the image field features to obtain field attributes of field information in the file image; and determining structured information included in the file to be identified according to the file type and the field attributes.

20. A structured information detection apparatus, characterized by comprising: Comprise: The second determining module is configured to determine a processing resource corresponding to the information identification service in response to a request for invoking the information identification service; The second processing module is configured to use the processing resource to perform the following steps: obtaining a file image of a file to be identified; determining a file type of the file to be identified based on the file image; obtaining category semantic features corresponding to the file type and image field features corresponding to the file image; performing field detection on the file image based on the category semantic features and the image field features to obtain field attributes of field information in the file image; and determining structured information included in the file to be identified according to the file type and the field attributes.

21. An electronic device, comprising: Comprise: The memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method for detecting structured information according to claim 19.

22. A method of detecting structured information, characterized by, Comprise: Obtaining an information identification request for an image to be identified, the image to be identified being a file image; determining structured information included in the image to be identified based on the information identification request, the structured information being determined based on a file type of the image to be identified and field attributes of field information in the image to be identified, the field attributes of the field information in the image to be identified being obtained by performing structured detection on the image to be identified, wherein the structured detection on the image to be identified comprises: obtaining category semantic features corresponding to the file type and image field features corresponding to the file image, and performing field detection on the file image based on the category semantic features and the image field features; displaying the structured information included in the image to be identified.

23. An apparatus for detecting structured information, characterized by Comprise: The third obtaining module is configured to obtain an information identification request for an image to be identified, the image to be identified being a file image; The third determining module is configured to determine the structured information included in the to-be-identified image based on the information identification request, the structured information being determined based on a file type of the to-be-identified image and a field attribute of field information in the to-be-identified image, the field attribute of the field information in the to-be-identified image being obtained by performing structured detection on the to-be-identified image, wherein the structured detection on the to-be-identified image comprises: obtaining a category semantic feature corresponding to the file type and an image field feature corresponding to the file image, and performing field detection on the file image based on the category semantic feature and the image field feature. The third displaying module is configured to display the structured information included in the to-be-identified image.

24. An electronic device, comprising: The method comprises: A memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method for detecting structured information according to claim 22.

Images