Data extraction method, device, equipment and storage medium based on image recognition

Abstract

The present application relates to the field of artificial intelligence and digital medical technology, and discloses a data extraction method, device, medium and equipment based on image recognition, wherein the method comprises: obtaining a target image, performing image recognition on the target image, and extracting a plurality of target text segments in the target image according to a paragraph format; performing semantic recognition on the target text segment through an ALBERT recognition model to obtain a semantic recognition result; annotating the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment; obtaining a first standard template, establishing a corresponding relationship between the first standard template and the first annotation information according to a preset rule, and filling the target text segment into the text filling position of the first standard template according to the corresponding relationship; achieving a structured arrangement of data, thereby improving the verification efficiency of medical contracts and reducing the error rate of the verification process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and digital medical technology, and in particular to a data extraction method, device, medium and equipment based on image recognition. Background Art

[0002] The launch of a medical project is usually based on the common goals of hospitals and pharmaceutical companies. Therefore, at the beginning of the project, it is necessary to agree on the subject of medical data authorization, the time, method, scope and application scenarios of use. Due to the wide variety of medical project scenarios and the lack of standardized specifications in the market, these contents are usually agreed upon by the participating parties through agreements or contracts.

[0003] However, the formats of self-agreed contracts are usually different. Medical data management personnel need to check and decompose the contents of these lengthy and complicated contracts through manual proofreading, maintain them in the data asset management of the maintenance organization, and spend huge time and cost to check them to ensure the compliance and security of medical data assets, resulting in low verification efficiency and high error rate. Summary of the invention

[0004] The main purpose of this application is to provide a data extraction method, device, medium and equipment based on image recognition, aiming to solve the technical problems of low efficiency and high error rate in the verification of medical contracts in the prior art.

[0005] In order to achieve the above-mentioned invention object, the present application proposes a data extraction method based on image recognition, the method comprising:

[0006] Acquire a target image, perform image recognition on the target image, and extract a plurality of target text segments in the target image according to a paragraph format;

[0007] Performing semantic recognition on the target text segment through the ALBERT recognition model to obtain a semantic recognition result;

[0008] Annotate the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment;

[0009] A first standard template is obtained, a corresponding relationship between the first standard template and the first annotation information is established according to a preset rule, and the target text segment is filled into a text filling position of the first standard template according to the corresponding relationship.

[0010] Furthermore, before the semantic recognition of the target text segment by the ALBERT recognition model, the method further includes:

[0011] Selecting a number of training keywords from the database, wherein the training keywords are common words in a specified application field;

[0012] Establishing an ALBERT initial model, wherein the ALBERT initial model has a built-in named entity recognition algorithm;

[0013] The ALBERT initial model is trained using the training keywords to obtain the ALBERT recognition model.

[0014] Furthermore, performing image recognition on the target image includes:

[0015] Performing layout analysis on the target image to identify text information and non-text information in the target image;

[0016] Describing the outline of the non-text information contained in the target image to obtain a non-text area;

[0017] After the non-text area is cropped, a cropped image containing the text information is obtained, and image recognition is performed on the cropped image.

[0018] Furthermore, the performing image recognition on the target image further includes:

[0019] Binarizing the target image to obtain a binary image;

[0020] Identify the paragraph outline in the binary image, and if the angle between two adjacent edges in the paragraph outline exceeds a preset angle threshold range, perform angle correction processing on the corresponding portion of the paragraph outline in the target image until the angle between two adjacent edges in the paragraph outline satisfies the angle threshold range, thereby obtaining an angle-corrected image;

[0021] Perform image recognition on the angle-corrected picture.

[0022] Furthermore, the performing image recognition on the target image further includes:

[0023] Numbering each of the image fragments and recording the arrangement position of the numbers;

[0024] Identify and delete blank fragments in the image fragments, restore the positions of the image fragments except the blank fragments according to the numbers, and perform image recognition on the restored image fragments.

[0025] Furthermore, performing semantic recognition on the target text segment to obtain a semantic recognition result includes:

[0026] Acquire a written time feature format set, wherein the written time feature format set includes a plurality of time formats used for written expression;

[0027] Characteristic information that matches the time format is identified in the target text segment, and the characteristic information is used as time information.

[0028] Furthermore, after performing image recognition on the target image, the method further includes:

[0029] Obtaining a signature image in the target image;

[0030] Extracting signature image features from the signature image through a neural network model;

[0031] The signature image features are compared with the standard image features of a preset standard signature. If the feature similarity between the signature image features and the standard image features is greater than a preset feature threshold, the content in the target image is determined to be valid. If the feature similarity between the signature image features and the standard image features is not greater than the preset feature threshold, the content in the target image is determined to be invalid.

[0032] The present application also proposes a data extraction device based on image recognition, comprising:

[0033] A text segment extraction module, used to obtain a target image, perform image recognition on the target image, and extract a number of target text segments in the target image according to a paragraph format;

[0034] A semantic recognition module performs semantic recognition on the target text segment through an ALBERT recognition model to obtain a semantic recognition result;

[0035] A first annotation module semantically annotates the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment;

[0036] The text reconstruction module is used to obtain a first standard template, establish a correspondence between the first standard template and the first annotation information according to a preset rule, and fill the target text segment into the text filling position of the first standard template according to the correspondence.

[0037] The present application also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0038] The present application also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0039] The image recognition-based data extraction method, device, medium and equipment of the present application obtain the data to be identified by acquiring the target image, so as to extract data from paper documents or electronic documents without word format, thereby improving versatility and expanding the scope of application; by performing image recognition on the target image and extracting the text content in segments according to the paragraph format in the target image, it is convenient to distinguish the independent meanings of different target text segments, and avoid the problem of semantic recognition errors caused by the mixing of text information with different meanings due to the inability to segment the text; by performing semantic recognition on each target text segment, a more accurate semantic recognition result is obtained, so as to distinguish the functional semantics of each target text segment, and the ALBERT model recognition can also realize cross-layer parameter sharing by contributing all parameters of all layers, reducing the amount of embedding calculation, thereby further improving the parameter calculation efficiency; the target text segment is annotated by the semantic recognition result, so as to realize accurate distinction of the attributes of each target text segment; each target text segment is converted from unstructured data to a structured arrangement corresponding to the first standard template according to the first standard template, so as to verify each target text segment, thereby improving the verification efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of a data extraction method based on image recognition according to an embodiment of the present application;

[0041] Figure 2 This is a flow chart of a data extraction method based on image recognition according to a specific implementation of the present application;

[0042] Figure 3 This is a schematic block diagram of the structure of a data extraction device based on image recognition according to an embodiment of the present application;

[0043] Figure 4 A schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] Reference Figure 1 In order to achieve the above-mentioned invention object, the present application proposes a data extraction method based on image recognition, the method comprising:

[0047] S1: Acquire a target image, perform image recognition on the target image, and extract a plurality of target text segments in the target image according to a paragraph format;

[0048] S2: Perform semantic recognition on the target text segment through the ALBERT recognition model to obtain a semantic recognition result;

[0049] S3: Annotate the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment;

[0050] S4: Acquire a first standard template, establish a correspondence between the first standard template and the first annotation information according to a preset rule, and fill the target text segment into the text filling position of the first standard template according to the correspondence.

[0051] This embodiment obtains the data to be identified by acquiring the target image, so as to extract data from paper documents or electronic documents without word format, thereby improving versatility and expanding the scope of application; by performing image recognition on the target image and extracting the text content in segments according to the paragraph format in the target image, it is convenient to distinguish the independent meanings of different target text segments, and avoid the problem of semantic recognition errors caused by the mixing of text information with different meanings due to the inability to segment the text; by performing semantic recognition on each target text segment, a more accurate semantic recognition result is obtained, so as to distinguish the functional semantics of each target text segment, and the ALBERT model recognition can also realize cross-layer parameter sharing by contributing all parameters of all layers, reducing the amount of embedding calculation, thereby further improving the parameter calculation efficiency; the target text segment is annotated by the semantic recognition result, so as to realize accurate distinction of the attributes of each target text segment; each target text segment is converted from unstructured data to a structured arrangement corresponding to the first standard template according to the first standard template, so as to verify each target text segment, thereby improving the verification efficiency and accuracy.

[0052] For step S1, this embodiment can acquire and process relevant data based on artificial intelligence technology. Wherein, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results; wherein, the above-mentioned relevant data can be medical contracts, medical agreements, etc., which are usually prepared by hospitals, pharmaceutical companies, patients and other participating parties without fixed templates; when the verification personnel need to verify one or more medical contracts or agreements, if the medical contract is a paper document, the paper contract or agreement can be pre-scanned into the computer, or the paper contract or agreement can be collected into the form of a picture and sent to the computer through a camera or other equipment. If the original contract or agreement is in the form of a pdf or picture state, an upload instruction can be directly initiated, and the contract or agreement in the pdf or picture state can be uploaded according to the preset upload window, so that the present invention can obtain the target picture according to the upload instruction, and the target picture is the contract or agreement in the form of a pdf or picture state. The present invention obtains the target image according to the upload instruction, thereby obtaining the data to be identified, so as to extract data from paper documents or electronic documents without word format, thereby improving versatility and expanding the scope of application.

[0053] After the target image is acquired, the text in the target image is extracted by image recognition methods such as OCR (Optical Character Recognition), thereby realizing the conversion from image to text. Since in actual contracts or agreements, different clauses are usually listed by segmentation, line division, labeling, etc., the present invention adopts a segmented extraction method to extract several target text segments according to the paragraph format in the target image; specifically, there is usually a blank space of two character widths at the beginning of a paragraph, and the beginning of a numbered paragraph is usually a combination of numbers and specific symbols, or a combination of letters and specific symbols, such as "1.", "1," "(1)," "1)," "a.", etc., that is, segmented extraction can be performed through such paragraph formats to obtain several target text segments; the present invention performs image recognition on the target image and extracts the text content segment by segment according to the paragraph format in the target image, so as to distinguish the independent meanings of different target text segments, and avoid the problem of semantic recognition errors caused by the inability to segment text and the mixing of text information with different meanings.

[0054] For step S2, since the formats of the original contracts or agreements are often not completely consistent, it is difficult to clarify the meaning of each target text segment by the position of the target text segment in the original contract or agreement, so it is necessary to perform semantic recognition on each target text segment; specifically, the ALBERT (ALite Bidirectional Encoder Representations from Transformers, simplified bidirectional encoder) recognition model can be used for semantic recognition. Since word embedding is to learn context-independent representations, and hidden layer embedding is to learn context-related representations, a large part of the representation ability of the recognition model comes from using the context to provide context-related representation signals for the learning process; therefore, separating the word embedding size E from the hidden layer size H can more efficiently utilize the overall model parameters. Generally speaking, H is much larger than E; however, for the ALBERT recognition model, it factorizes the word embedding parameters into two small matrices, and no longer directly maps the one-hot vector to the hidden space of size H, but first maps them to a low-dimensional word embedding space E, and then Then map it to the hidden space; through the above, the ALBERT recognition model can reduce the word embedding parameters from O(V×H) to O(V×E+E×H). For example, taking ALBert_xxlarge as an example, if V=30000, H=4096, and E=128, the total number of word embedding parameters under the original calculation method is V*H=30000*4096=123 million parameters, and the total number of word embedding parameters under the ALBERT recognition model is V*E+E*H=30000*128+128*4096=3.84 million+520,000=4.36 million. The parameters related to word embedding are 28 times before the change, which shows that when H is much larger than E, the number of parameters is significantly reduced. In addition, the ALBERT model recognition model can also realize cross-layer parameter sharing by contributing all parameters of all layers, reducing the amount of Embedding calculations, thereby further improving the efficiency of parameter calculations.

[0055] For step S3, in a medical contract or agreement, there are usually the following types of information that need to be checked in particular: authorized subject, authorized time, authorized data, and authorized scope of use. After the semantics of each target text segment are identified, they can be annotated according to their respective semantics; illustratively, if a target text segment is identified to contain words such as "company", "limited liability company", "institution", "hospital", "Medical Insurance Bureau", or similar words, then the target text segment can be determined to be an authorized subject, and the first annotation information of the target text segment is set to "authorized subject"; if words such as "authorized area", "within the province..." or similar words are identified, then the target text field can be determined to be an authorized scope, and the first annotation information of the target text segment is set to "authorized area"; this embodiment annotates the target text segment through the semantic recognition result, thereby achieving accurate distinction of the attributes of each target text segment.

[0056] For step S4, the first standard template is a template that facilitates information verification by computers and verification personnel. The first standard template generally includes text fill-in bits for the authorized subject, authorized time, authorized data, and authorized use scope, and the first annotation information may include the authorizer, the authorized person, the authorized time point, the authorized time period, the authorized laboratory data, the authorized medical data, the authorized instrument data, the authorized use area, the authorized use hospital, etc. Based on the first standard template and the first annotation information, the preset rule may be to correspond the authorizer and the authorized person to the authorized subject, the authorized time point and the authorized time period to the authorized time, the authorized laboratory data, the authorized medical data, and the authorized instrument data to the authorized data, and the authorized use area and the authorized use hospital to the authorized use scope, thereby obtaining a corresponding relationship between the first standard template and the first annotation information;

[0057] Therefore, after obtaining the first annotation information corresponding to the target text segment, the target text segment corresponding to the first annotation information can be filled into the text filling position of the first standard template based on the corresponding relationship established by the preset rules, so that each target text segment is converted from unstructured data to a structured arrangement corresponding to the first standard template. It can be understood that after unifying contracts or agreements of different formats into the first standard template, for the computer, it can directly perform text verification one by one according to the positional relationship in the first standard template, and for the corresponding verification personnel, it can also intuitively perform text review according to the positional relationship in the first standard template; for example, if the arrangement order of each information in the first standard template is "1. Authorized subject: xxx; 2. Authorization time: xxxx.xx.xx; 3. Authorization data: xxx; 4. Authorized use scope: xxx~xxx"; then the computer can identify them one by one in the order of "authorized subject-authorization time-authorization data-authorized use scope", avoiding the "authorized use scope" caused by the inconsistent format. The verification personnel can also review them one by one in the order of "authorized subject-authorization time-authorized data-authorized scope of use". Because people are prone to thinking inertia when processing large amounts of data, structuring the data in a unified order can make use of people's thinking inertia, improve the review efficiency of the verification personnel, and reduce the learning cost; after the target text segment is filled in the text filling position of the first standard template, the data can be stored in JSON format, so that after the verification is completed, in response to the confirmation operation received by the system, the JSON format metadata of the batch of data is automatically obtained, and the data usage scope, time and application scenarios are controlled through the data middle platform asset management function.

[0058] In one embodiment, before performing semantic recognition S4 on the target text segment by using the ALBERT recognition model, the method further includes:

[0059] S401: Selecting a number of training keywords from a database, wherein the training keywords are common words in a specified application field;

[0060] S402: Establishing an ALBERT initial model, wherein the ALBERT initial model has a built-in named entity recognition algorithm;

[0061] S403: Using the training keywords to train the ALBERT initial model to obtain the ALBERT recognition model.

[0062] In this embodiment, the ALBERT initial model is trained by presetting keywords in the medical field to obtain the ALBERT recognition model, so that the ALBERT recognition model is more suitable for semantic recognition of medical contracts or agreements; the semantics or phrase similarities of these target text segments are analyzed through the built-in named entity recognition algorithm of the ALBERT recognition model, so as to obtain more accurate semantic recognition results.

[0063] For step S401, in order to obtain a more targeted and professional ALBERT recognition model, this embodiment adopts a method of selecting common words in the specified application field as training keywords. For example, if the specified application field is the vehicle field, the training keywords are common words related to vehicles; and when the specified application field is the medical contract or agreement field, since there are many ways to express the scope of data authorization in the actual medical contract or agreement writing terms, the phrases commonly used in the medical contract or agreement field such as "data authorization purpose" and "data is used for xx project" can be selected as the training keywords of the application field. In addition, the expression of the authorized subject usually also has different forms of common words such as "company", "limited company", "institution", "hospital", "medical insurance bureau", etc. Therefore, words such as "company", "limited company", "institution", "hospital", "medical insurance bureau" can be selected as the training keywords of the application field. In order to improve the recognition accuracy of such highly professional and irregular words, this embodiment uses common words in the specified application field as training keywords, and pre-trains the model through the training keywords. In a specific implementation, the common words in the above-mentioned designated application field may be manually input into the above-mentioned database, or may be common word combinations of medical contracts or agreements stored in the database by the cloud server.

[0064] For step S402, after obtaining ALBERT recognition through the above keyword training, the attributes of the above target text segment can be identified according to the position and content of the keyword recognition, and the semantics or phrase similarity of these target text segments can be analyzed through the method of named entity recognition (NER), and finally the text is structured. Named entity recognition is a basic function in NLP that can be used for information extraction and syntactic analysis.

[0065] In one embodiment, referring to Figure 2 The performing image recognition S1 on the target image includes:

[0066] S11: Performing layout analysis on the target image to identify text information and non-text information in the target image;

[0067] S12: Outlining the non-text information contained in the target image to obtain a non-text area;

[0068] S13: After cropping the non-text area, a cropped image containing the text information is obtained, and image recognition is performed on the cropped image.

[0069] This embodiment crops the non-text area so that the subsequent recognition model only recognizes the text information, thereby improving the recognition efficiency, eliminating the interference of the non-text information, and improving the recognition accuracy.

[0070] Regarding step S11, in daily contracts or agreements, sometimes the company logo, company name, and other information that is irrelevant to the main content of the contract or agreement will be marked on the cover, header, footer, and other locations. If the header, footer, etc. in each target image are identified, it will lead to unnecessary text extraction and increase the amount of text calculation. Therefore, before text extraction, this embodiment first treats the cover, header, and footer as non-text information based on the layout analysis results of the target image, and treats the part located in the middle of the layout and far away from the four sides of the target image as text information. In a specific implementation, the border distance can be preset according to the image size. Taking the A4 size as an example, the positions 2 cm away from the upper and lower borders can be used as headers and footers, respectively. Therefore, the size of the target image can be obtained through layout analysis. When the size is 210mm×297mm, the target image is determined to be A4 size. When the size is 148mm×210mm, the target image is determined to be A5 size, and so on. Finally, according to the size of the target image and the border distance, the headers and footers close to the upper and lower borders are identified as non-text information.

[0071] For step S12, the portion where the non-text information is located is divided into outlines. For example, the areas containing all non-text information, i.e., the header and footer parts, can be selected separately by a rectangular selection box with a width equal to the border distance and a length equal to the width of the target image, and the divided non-text areas of the header and footer are cropped, so that the remaining cropped images containing the text information have a higher proportion of text information, thereby improving the effectiveness of recognition, eliminating the interference of non-text information, and improving the accuracy of recognition.

[0072] In one embodiment, the performing image recognition S1 on the target image further includes:

[0073] S14: performing binarization processing on the target image to obtain a binarized image;

[0074] S15: Identify the paragraph outline in the binary image, and if the angle between two adjacent edges in the paragraph outline exceeds a preset angle threshold range, perform angle correction processing on the corresponding part of the paragraph outline in the target image until the angle between two adjacent edges in the paragraph outline satisfies the angle threshold range, thereby obtaining an angle-corrected image;

[0075] S16: Performing image recognition on the angle correction picture.

[0076] Before image recognition, this embodiment first performs binarization processing on the target image to make the image simpler, reduce the amount of data, highlight the outline of the target with higher attention, and obtain text with less deformation through angle correction, thereby improving the accuracy of subsequent image recognition.

[0077] For step S14, since the contract or agreement is usually printed in paper form and becomes effective after being signed or stamped by the parties involved, when the paper version of the contract or agreement needs to be verified, it is often necessary to take a photo of it and upload it or scan it into a computer. However, due to the shooting environment and the quality of the scanning machine, the picture received by the computer has unclear content, or the shooting angle is tilted, which makes it difficult to clearly identify the text. In order to solve the above problems, this embodiment pre-processes the target image before image recognition. Specifically, the target image is first binarized to make the image simpler, reduce the amount of data, and highlight the outline of the target with higher attention. After the binarization process, since the position where the text exists is usually black and the position where the text does not exist is usually white, the paragraph outline can be recognized by the color boundary.

[0078] For step S15, when the shooting angle is too inclined, the paragraph outline of the captured text part may present a trapezoid, parallelogram or other irregular quadrilateral. At this time, the text needs to be angle corrected so that the recognition model can recognize text with a more correct angle; specifically, the closer the angle between two adjacent sides is to 90°, the better the recognition effect. Therefore, the preset angle threshold range can be 85° to 95°. When the angle is less than 85°, the top of the angle can be used as the center, and the two endpoints of the quadrilateral adjacent to the top can be stretched outward until the degree of the angle is not less than 85°. When the angle is greater than 95°, the two endpoints of the quadrilateral adjacent to the top of the angle can be fixed, and the top of the angle can be pushed inward until the degree of the angle is not greater than 95°, thereby completing the angle correction and obtaining text with a smaller degree of deformation, thereby improving the accuracy of subsequent image recognition.

[0079] In one embodiment, the performing image recognition S1 on the target image further includes:

[0080] S17: Cut the target image into pieces to obtain a number of image fragments;

[0081] S18: numbering each of the image fragments, and recording the arrangement position of the number;

[0082] S19: Identify and delete blank fragments in the image fragments, restore the positions of the image fragments except the blank fragments according to the serial numbers, and perform image recognition on the restored image fragments.

[0083] This embodiment reduces the amount of calculation for subsequent image recognition by cutting the target image and removing blank fragments.

[0084] For step S18, before image recognition, the target image is first cut into rows and columns to obtain several image fragments. In order to avoid recognition failure caused by disrupting the order of the image fragments, it is necessary to record the fragment numbers and the arrangement positions of the numbers. For example, if nine image fragments are obtained by row and column cutting, they can be recorded according to the arrangement positions that the three columns of the first row are "123", the three columns of the second row are "456", and the three columns of the third row are "789".

[0085] For step S19, the image fragments without pixel points are regarded as blank fragments and deleted to avoid the problem of low recognition efficiency caused by the image being too large. After deleting the blank fragments, the positions of the remaining image fragments are restored according to the above numbers. Specifically, if the image fragments numbered 5, 6, 8 and 9 are blank fragments, the remaining image fragments are restored according to the positions where the three columns of the first row are "123", the first column of the second row is "4", and the first column of the third row is "7".

[0086] In one embodiment, the semantic recognition of the target text segment to obtain the semantic recognition result S3 includes:

[0087] S31: Acquire a written time feature format set, wherein the written time feature format set includes a plurality of time formats used for written expression;

[0088] S32: identifying feature information that matches the time format in the target text segment, and using the feature information as time information.

[0089] This embodiment identifies matching feature information in a target text segment through a written time feature format set, thereby achieving accurate extraction of time information in the target text segment.

[0090] For step S31, the writing format of time information is often different from other conversations containing detailed semantics. In a contract, there may be "2020.07.01", or there may be "2020.7.1", "2020-07-01", "8th March, 2020", "du8 March, 2020", etc. Therefore, in order to avoid confusing the time information with the semantics of other sentences, this embodiment first obtains a written time feature format set, which includes the above-mentioned various time formats for written expression. For example, it can be in the format of "year+separator+month+separator+date", "date+month+separator+year".

[0091] For step S32, when it is identified that there is text satisfying any of the above formats in the target image, the text is determined to be time information.

[0092] In one embodiment, after performing image recognition S1 on the target image, the method further includes:

[0093] S121: Acquire the signature image in the target image;

[0094] S122: extracting signature image features from the signature image through a neural network model;

[0095] S123: Compare the signature image features with the standard image features of a preset standard signature. If the feature similarity between the signature image features and the standard image features is greater than a preset feature threshold, determine that the content in the target image is valid. If the feature similarity between the signature image features and the standard image features is not greater than the preset feature threshold, determine that the content in the target image is invalid.

[0096] This embodiment compares the signature image in the target image with the features of the standard signature to determine whether the signature in the target image is the signature of the person himself, thereby improving the security of the medical contract or agreement.

[0097] In step S121, the signature image in the target image is obtained through image recognition such as OCR (Optical Character Recognition).

[0098] For step S122, the signature image features in the signature image are extracted through a convolutional neural network model to generate a feature vector sequence.

[0099] For step S123, when signature verification is required, the real signatures of the parties to the medical contract or agreement corresponding to the target image can be obtained as standard signatures, and the standard signatures can be pre-extracted through a convolutional neural network model to obtain standard image features, so as to facilitate comparison between the standard image features and the signature image features. When the similarity between the above two features is less than a preset threshold, it is determined that the signature is not the signature of the person himself, that is, there is a risk of forgery in the medical contract or agreement, and therefore the content in the target image is determined to be invalid, and an alarm is issued.

[0100] Reference Figure 3 , the present application also proposes a data extraction device based on image recognition, comprising:

[0101] The text segment extraction module 100 is used to obtain a target image, perform image recognition on the target image, and extract a plurality of target text segments in the target image according to a paragraph format;

[0102] Semantic recognition module 200, semantic recognition is performed on the target text segment through the ALBERT recognition model to obtain a semantic recognition result;

[0103] A first annotation module 300 semantically annotates the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment;

[0104] The text reconstruction module 400 is used to obtain a first standard template, establish a correspondence between the first standard template and the first annotation information according to a preset rule, and fill the target text segment into the text filling position of the first standard template according to the correspondence.

[0105] This embodiment obtains the data to be identified by acquiring the target image, so as to extract data from paper documents or electronic documents without word format, thereby improving versatility and expanding the scope of application; by performing image recognition on the target image and extracting the text content in segments according to the paragraph format in the target image, it is convenient to distinguish the independent meanings of different target text segments, and avoid the problem of semantic recognition errors caused by the mixing of text information with different meanings due to the inability to segment the text; by performing semantic recognition on each target text segment, a more accurate semantic recognition result is obtained, so as to distinguish the functional semantics of each target text segment, and the ALBERT model recognition can also realize cross-layer parameter sharing by contributing all parameters of all layers, reducing the amount of embedding calculation, thereby further improving the parameter calculation efficiency; the target text segment is annotated by the semantic recognition result, so as to realize accurate distinction of the attributes of each target text segment; each target text segment is converted from unstructured data to a structured arrangement corresponding to the first standard template according to the first standard template, so as to verify each target text segment, thereby improving the verification efficiency and accuracy.

[0106] In one embodiment, a model training module 500 is further included, which is used to:

[0107] Selecting a number of training keywords from the database, wherein the training keywords are common words in a specified application field;

[0108] Establishing an ALBERT initial model, wherein the ALBERT initial model has a built-in named entity recognition algorithm;

[0109] The ALBERT initial model is trained using the training keywords to obtain the ALBERT recognition model.

[0110] In one embodiment, the text segment extraction module 100 is further used for:

[0111] Performing layout analysis on the target image to identify text information and non-text information in the target image;

[0112] Describing the outline of the non-text information contained in the target image to obtain a non-text area;

[0113] After the non-text area is cropped, a cropped image containing the text information is obtained, and image recognition is performed on the cropped image.

[0114] In one embodiment, the text segment extraction module 100 is further used for:

[0115] Binarizing the target image to obtain a binary image;

[0116] Identify the paragraph outline in the binary image, and if the angle between two adjacent edges in the paragraph outline exceeds a preset angle threshold range, perform angle correction processing on the corresponding portion of the paragraph outline in the target image until the angle between two adjacent edges in the paragraph outline satisfies the angle threshold range, thereby obtaining an angle-corrected image;

[0117] Perform image recognition on the angle-corrected picture.

[0118] In one embodiment, the text segment extraction module 100 is further used for:

[0119] Cutting the target image to obtain a number of image fragments;

[0120] Numbering each of the image fragments and recording the arrangement position of the numbers;

[0121] Identify and delete blank fragments in the image fragments, restore the positions of the image fragments except the blank fragments according to the numbers, and perform image recognition on the restored image fragments.

[0122] In one embodiment, the text segment extraction module 100 is further used for:

[0123] Acquire a written time feature format set, wherein the written time feature format set includes a plurality of time formats used for written expression;

[0124] identifying feature information in the target text segment that matches the time format;

[0125] The characteristic information is identified as time information.

[0126] In one embodiment, a signature checking module 600 is further included, which is used to:

[0127] Obtaining a signature image in the target image;

[0128] Extracting signature image features from the signature image through a neural network model;

[0129] The signature image features are compared with the standard image features of a preset standard signature. If the feature similarity between the signature image features and the standard image features is greater than a preset feature threshold, the content in the target image is determined to be valid. If the feature similarity between the signature image features and the standard image features is not greater than the preset feature threshold, the content in the target image is determined to be invalid.

[0130] Reference Figure 4In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a data extraction method based on image recognition. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data extraction method based on image recognition is implemented. The data extraction method based on image recognition includes: obtaining a target image, performing image recognition on the target image, and extracting a plurality of target text segments in the target image according to a paragraph format; performing semantic recognition on the target text segment through an ALBERT recognition model to obtain a semantic recognition result; annotating the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment; obtaining a first standard template, establishing a corresponding relationship between the first standard template and the first annotation information according to a preset rule, and filling the target text segment into a text filling position of the first standard template according to the corresponding relationship.

[0131] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a data extraction method based on image recognition is implemented, including the steps of: acquiring a target image, performing image recognition on the target image, and extracting a plurality of target text segments in the target image according to a paragraph format; performing semantic recognition on the target text segment through an ALBERT recognition model to obtain a semantic recognition result; annotating the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment; acquiring a first standard template, establishing a corresponding relationship between the first standard template and the first annotation information according to a preset rule, and filling the target text segment into the text filling position of the first standard template according to the corresponding relationship.

[0132] In the above-mentioned data extraction method based on image recognition, this embodiment obtains the data to be identified by acquiring the target image, so as to extract data from paper documents or electronic documents without word format, thereby improving versatility and expanding the scope of application; by performing image recognition on the target image and extracting the text content in segments according to the paragraph format in the target image, it is convenient to distinguish the independent meanings of different target text segments, and avoid the problem that the text information with different meanings is mixed together due to the inability to segment the text, resulting in semantic recognition errors; by performing semantic recognition on each target text segment, a more accurate semantic recognition result is obtained, so as to distinguish the functional semantics of each target text segment, and the ALBERT model recognition can also realize cross-layer parameter sharing by contributing all parameters of all layers, thereby reducing the amount of embedding calculation, thereby further improving the parameter calculation efficiency; the target text segment is annotated by the semantic recognition result, so as to realize accurate distinction of the attributes of each target text segment; each target text segment is converted from unstructured data to a structured arrangement corresponding to the first standard template according to the first standard template, so as to verify each target text segment, thereby improving the verification efficiency and accuracy.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0134] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0135] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data extraction method based on image recognition, characterized in that: The method comprises: Acquire a target image, perform image recognition on the target image, and extract a plurality of target text segments in the target image according to a paragraph format; Performing semantic recognition on the target text segment through the ALBERT recognition model to obtain a semantic recognition result; Annotate the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment; Acquire a first standard template, establish a correspondence between the first standard template and the first annotation information according to a preset rule, and fill the target text segment into a text filling position of the first standard template according to the correspondence; The performing image recognition on the target image further includes: Cutting the target image to obtain a number of image fragments; Numbering each of the image fragments and recording the arrangement position of the numbers; Identify and delete blank fragments in the picture fragments, restore the positions of the picture fragments except the blank fragments according to the serial numbers, and perform image recognition on the restored picture fragments; After performing image recognition on the target image, the method further includes: Obtaining a signature image in the target image; Extracting signature image features from the signature image through a neural network model; The signature image features are compared with the standard image features of a preset standard signature. If the feature similarity between the signature image features and the standard image features is greater than a preset feature threshold, the content in the target image is determined to be valid. If the feature similarity between the signature image features and the standard image features is not greater than the preset feature threshold, the content in the target image is determined to be invalid.

2. The data extraction method based on image recognition according to claim 1, characterized in that: Before the semantic recognition of the target text segment is performed by the ALBERT recognition model, the method further includes: Selecting a number of training keywords from the database, wherein the training keywords are common words in a specified application field; Establishing an ALBERT initial model, wherein the ALBERT initial model has a built-in named entity recognition algorithm; The ALBERT initial model is trained using the training keywords to obtain the ALBERT recognition model.

3. The data extraction method based on image recognition according to claim 1, characterized in that: The performing image recognition on the target image includes: Performing layout analysis on the target image to identify text information and non-text information in the target image; Describing the outline of the non-text information contained in the target image to obtain a non-text area; After the non-text area is cropped, a cropped image containing the text information is obtained, and image recognition is performed on the cropped image.

4. The data extraction method based on image recognition according to claim 1, characterized in that: The performing image recognition on the target image further includes: Binarizing the target image to obtain a binary image; Identify the paragraph outline in the binary image, and if the angle between two adjacent edges in the paragraph outline exceeds a preset angle threshold range, perform angle correction processing on the corresponding portion of the paragraph outline in the target image until the angle between two adjacent edges in the paragraph outline satisfies the angle threshold range, thereby obtaining an angle-corrected image; Perform image recognition on the angle-corrected picture.

5. The data extraction method based on image recognition according to claim 1, characterized in that: The performing semantic recognition on the target text segment to obtain a semantic recognition result includes: Acquire a written time feature format set, wherein the written time feature format set includes a plurality of time formats used for written expression; Characteristic information that matches the time format is identified in the target text segment, and the characteristic information is used as time information.

6. A data extraction device based on image recognition, characterized in that: include: A text segment extraction module, used to obtain a target image, perform image recognition on the target image, and extract a number of target text segments in the target image according to a paragraph format; A semantic recognition module performs semantic recognition on the target text segment through an ALBERT recognition model to obtain a semantic recognition result; A first annotation module annotates the target text segment according to the semantic recognition result to obtain first annotation information corresponding to the target text segment; A text reconstruction module, used for acquiring a first standard template, establishing a correspondence between the first standard template and the first annotation information according to a preset rule, and filling the target text segment into a text filling position of the first standard template according to the correspondence; The text segment extraction module is further used for: Cutting the target image to obtain a number of image fragments; Numbering each of the image fragments and recording the arrangement position of the numbers; Identify and delete blank fragments in the picture fragments, restore the positions of the picture fragments except the blank fragments according to the serial numbers, and perform image recognition on the restored picture fragments; The data extraction device also includes a signature verification module, which is used to: Obtaining a signature image in the target image; Extracting signature image features from the signature image through a neural network model; The signature image features are compared with the standard image features of a preset standard signature. If the feature similarity between the signature image features and the standard image features is greater than a preset feature threshold, the content in the target image is determined to be valid. If the feature similarity between the signature image features and the standard image features is not greater than the preset feature threshold, the content in the target image is determined to be invalid.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images