An information extraction method and device, electronic equipment and storage medium

Abstract

The present application belongs to the technical field of data extraction, and discloses an information extraction method and device, an electronic device and a storage medium. The present application adopts a machine recognition and processing mode to automatically extract the content in pictures and documents, and performs error correction processing on the extracted content after extraction, so as to obtain an information keyword set with high accuracy. Compared with manual extraction, the present method is not only efficient, but also reduces labor costs, and is suitable for large-scale application and promotion in the field of information extraction technology.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to an information extraction method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of business needs, the extraction of data from documents and images is increasingly appearing in daily program development and application processes. This includes extracting content from program development requirement documents, development summary documents, development progress documents, or images to obtain program development requirements, summaries, and progress information, enabling developers to take appropriate measures to complete development work on time and according to requirements. Currently, the extraction of data from these documents and images mostly relies on manual identification, which not only requires a large amount of manpower but is also very inefficient. Therefore, providing a method for automatically extracting data from documents and images has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this invention is to provide an information extraction method, apparatus, electronic device, and storage medium to solve the problems of high manpower requirements and low reading efficiency in the prior art, which relies on manual reading of data from images and documents.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] Firstly, an information extraction method is provided, including:

[0006] Obtain the target file, wherein the target file includes documents and images;

[0007] The target file is subjected to information recognition processing to obtain the content information of the target file, and the content information is used as the source data of the target file.

[0008] Using regular expression matching, information extraction processing is performed on the source data to obtain a preliminary information set of the target file, wherein the preliminary information set contains multiple key fields;

[0009] The preliminary information set is subjected to data error correction processing to obtain the information keyword set of the target file after the data error correction processing.

[0010] Based on the above-disclosed content, after obtaining the target file, this invention first performs information recognition processing on the target file to identify the data content within the target file, which serves as the source data of the target file. Next, this invention performs information extraction processing on the source data to obtain the key content fields, thus forming a preliminary information set. Then, to ensure the accuracy of data extraction, this invention also performs data error correction processing on the preliminary information set to remove erroneous data. After data error correction processing, the information keyword set of the target file can be obtained, thereby completing the extraction of key content from the target file. Therefore, through the aforementioned design, this invention uses machine recognition and processing technology to automatically extract content from images and documents. Compared to manual extraction, it is not only more efficient but also reduces labor costs, making it suitable for large-scale application and promotion in the field of information extraction technology.

[0011] In one possible design, each key field in the preliminary information set corresponds to a file content attribute. The preliminary information set undergoes data correction processing to obtain the information keyword set of the target file, which includes:

[0012] Based on the preliminary information set, at least one field cluster to be corrected is identified, wherein any field cluster to be corrected contains multiple key fields, and the file content attributes corresponding to each key field contained in any field cluster to be corrected are the same.

[0013] Obtain the credibility scoring model, and input the key fields in each field cluster to be corrected into the credibility scoring model to obtain the credibility score of each key field in each field cluster to be corrected.

[0014] From each cluster of fields to be corrected, the key fields with the highest confidence scores are selected as replacement fields for each cluster of fields to be corrected.

[0015] For any of the fields to be corrected, the key fields in the preliminary information set that belong to any of the fields to be corrected are updated to the replacement fields of any of the fields to be corrected, so that after all the key fields in the preliminary information set that belong to each of the fields to be corrected are updated, the error correction information set is obtained.

[0016] The error correction information set is subjected to data verification processing to obtain the information keyword set after data verification processing.

[0017] In one possible design, the method further includes:

[0018] The source data is segmented to obtain the segmentation set of the target file;

[0019] Using each segmentation field in the segmentation set as input and the credibility score of each segmentation field in the target file as output, an initial scoring model is trained to obtain the credibility scoring model after training. The initial scoring model includes an SVM model or a random forest model.

[0020] In one possible design, the error correction information set undergoes data validation processing to obtain the information keyword set after data validation processing, including:

[0021] For any key field in the error correction information set, the standard length of the key field is determined based on the file content attribute corresponding to the key field.

[0022] Determine whether the actual length of any of the key fields meets the standard length.

[0023] If the actual length of any of the key fields is less than the standard length, then the regular expression matching method is used again to extract information from the source data until the actual length of any of the key fields meets the standard length, thus obtaining the information keyword set.

[0024] In one possible design, if the actual length of any of the key fields is greater than the standard length, the method further includes:

[0025] Determine whether any of the key fields contains special characters;

[0026] If so, then use regular expressions to remove special characters from any of the key fields, so that after all special characters in all key fields within the error correction information set have been removed, the information keyword set is obtained.

[0027] In one possible design, before performing information recognition processing on the target file, the method further includes:

[0028] The target file is subjected to anti-counterfeiting verification processing to obtain the anti-counterfeiting verification result;

[0029] If the anti-counterfeiting verification result indicates that the document is forged, a document forged prompt message will be generated;

[0030] Accordingly, after obtaining the set of information keywords for the target file, the method further includes:

[0031] The set of information keywords is associated with the document forgery warning information and stored in the database.

[0032] In one possible design, the target file is subjected to anti-counterfeiting verification processing to obtain the anti-counterfeiting verification result, including:

[0033] If the target file is an image, then the first file attribute information of the target file is read, and it is determined whether there is sensitive information in the first file attribute information;

[0034] If so, the NRSS detection algorithm is used to perform anti-counterfeiting detection on the target file, and the anti-counterfeiting detection score of the target file is obtained;

[0035] Determine whether the anti-counterfeiting detection score is greater than a preset threshold;

[0036] If so, the anti-counterfeiting verification result is generated as a document forgery;

[0037] If the target file is a document, then the second file attribute information of the target file is read, wherein the second file attribute information includes the creation time and creation location;

[0038] Determine whether the second file attribute information is abnormal;

[0039] If so, the anti-counterfeiting verification result is generated as a document forgery.

[0040] Secondly, an information extraction device is provided, comprising:

[0041] A data acquisition unit is used to acquire target files, wherein the target files include documents and images;

[0042] An information recognition unit is used to perform information recognition processing on the target file to obtain the content information of the target file, and use the content information as the source data of the target file;

[0043] The information extraction unit is used to perform information extraction processing on the source data using a regular expression matching method to obtain a preliminary information set of the target file, wherein the preliminary information set contains multiple key fields;

[0044] The information correction unit is used to perform data correction processing on the preliminary information set so as to obtain the information keyword set of the target file after the data correction processing.

[0045] Thirdly, another information extraction device is provided. Taking an electronic device as an example, it includes a memory, a processor, and a transceiver that are sequentially and communicatively connected. The memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the information extraction method as described in the first aspect or any possible design of the first aspect.

[0046] Fourthly, a storage medium is provided, on which instructions are stored, which, when executed on a computer, perform the information extraction method as described in the first aspect or any possible design of the first aspect.

[0047] Fifthly, a computer program product containing instructions is provided, which, when executed on a computer, cause the computer to perform the information extraction method as described in the first aspect or any possible design of the first aspect.

[0048] Beneficial effects:

[0049] (1) After obtaining the target file, the present invention first performs information recognition processing on the target file to identify the data content in the target file and use it as the source data of the target file; then, the present invention performs information extraction processing on the source data to obtain the key content fields in the source data, thereby forming a preliminary information set; then, in order to ensure the accuracy of data extraction, the present invention also performs data error correction processing on the preliminary information set to remove erroneous data in the preliminary information set. Thus, after the data error correction processing, the information keyword set of the target file can be obtained, thereby completing the extraction of key content in the target file; thus, through the aforementioned design, the present invention uses machine recognition and processing technology to automatically extract the content in images and documents. Compared with manual extraction, it is not only more efficient, but also reduces labor costs, and is suitable for large-scale application and promotion in the field of information extraction technology. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the steps of the information extraction method provided in an embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of the structure of the information extraction device provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0054] It should be understood that although the terms first, second, etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, without departing from the scope of the exemplary embodiments of the invention.

[0055] It should be understood that the term "and / or" that may appear in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" that may appear in this document describes another relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the related objects before and after it are in an "or" relationship.

[0056] Example:

[0057] See Figure 1 As shown, the information extraction method provided in this embodiment uses machine recognition and processing to automatically extract content from images and documents. After extraction, the extracted content is corrected to obtain a set of information keywords with high accuracy. Thus, compared with manual extraction, this method is not only more efficient but also reduces labor costs, making it suitable for large-scale application and promotion in the field of information extraction technology. In this embodiment, the method can be run on the information extraction end, which can be, but is not limited to, a personal computer (PC), tablet computer, or smartphone. It is understood that the aforementioned execution subject does not constitute a limitation on the embodiment of this application. Accordingly, the operation steps of this method can be, but are not limited to, the steps S1 to S4 below.

[0058] S1. Obtain the target file, wherein the target file includes documents and images; in specific applications, when the target file is an image, the data extraction from the image is not limited to Kafka tasks, that is, subsequent steps can be executed in the form of Kafka tasks; if the target file is a document, a file factory can be constructed to extract data from the document; of course, the aforementioned data extraction and processing methods are all commonly used tools in existing program development, and their principles will not be elaborated here.

[0059] After obtaining the target file, to ensure the authenticity of the file, this embodiment also includes an anti-counterfeiting detection step, which may include, but is not limited to, the steps shown below:

[0060] Step 1: Perform anti-counterfeiting verification on the target file to obtain the anti-counterfeiting verification result. In specific implementation, different anti-counterfeiting verification methods can be adopted according to the type of target file. Optionally, when the target file is an image, the NRSS detection algorithm and the EXIF ​​detection method (a method that uses file metadata to determine the authenticity of the file) can be used to achieve anti-counterfeiting verification. Specifically, the anti-counterfeiting detection process of the image can be, but is not limited to, the following steps A to D.

[0061] A. If the target file is an image, then read the first file attribute information of the target file and determine whether there is sensitive information in the first file attribute information. In this embodiment, an EXIF ​​detection tool can be used, but is not limited to, to read the first file attribute information of the target file. For example, the first file attribute information can include, but is not limited to, the program information used by the file. At the same time, sensitive information can include, but is not limited to, Adobe Photoshop, ACDsystems, meitu.com, etc. In this way, it can be determined whether the target file has been suspected of being modified based on the first file attribute information. Furthermore, if the aforementioned first file attribute information contains one or more of the aforementioned sensitive information, then it is determined that the target file has been suspected of being modified. At this time, the next step of detection needs to be performed, that is, step B below is executed. Otherwise, it can be determined that the target file has passed the anti-counterfeiting verification.

[0062] B. If so, the NRSS detection algorithm is used to perform anti-counterfeiting detection on the target file, and the anti-counterfeiting detection score of the target file is obtained. In specific applications, the NRSS detection algorithm is a method for detecting the non-reference structural sharpness of an image. It mainly performs low-pass filtering on the image first; then extracts the gradient information of the image; and then uses the image gradient information to calculate the structural sharpness (that is, the anti-counterfeiting detection score). In this way, based on the structural sharpness, it can be determined whether the target file has been modified. The judgment process is as shown in step C below.

[0063] C. Determine whether the anti-counterfeiting detection score is greater than a preset threshold. In specific applications, the preset threshold can be set according to the actual application. In this embodiment, it is preferably 20. Thus, when the anti-counterfeiting detection score is greater than 20, it is determined that the target file has been modified and its anti-counterfeiting verification result is that the file is counterfeit. Otherwise, it is determined that it has not been modified. As shown in step D below.

[0064] D. If so, then the anti-counterfeiting verification result is generated as document forgery.

[0065] Therefore, by following steps A to D above, the anti-counterfeiting verification process for the image can be completed.

[0066] Similarly, if the target file is a document, then the following steps E to G are used to complete the anti-counterfeiting detection.

[0067] E. If the target file is a document, then read the second file attribute information of the target file, wherein the second file attribute information includes the creation time and creation location; in specific applications, the creation location may include, but is not limited to, latitude and longitude, GPS coordinates, etc.; in this way, the anti-counterfeiting verification of the document can be performed based on the second file attribute information, wherein the anti-counterfeiting verification process of the document may be, but is not limited to, as shown in step F below.

[0068] F. Determine whether the second file attribute information is abnormal; in this embodiment, for example, but not limited to, first obtaining the upload address of the target file, and then determining whether the creation position in the second file attribute information has changed relative to the upload address; if it has changed, it is determined that the target file has been modified, and its anti-counterfeiting verification result is file forgery, that is, to execute the following step G; otherwise, it is determined that it has not been modified.

[0069] G. If so, then the anti-counterfeiting verification result is generated as a document forgery.

[0070] Thus, by going through the aforementioned steps A to G, the anti-counterfeiting verification of the target file can be completed. If the anti-counterfeiting verification result is that the file is counterfeit, then a file counterfeiting prompt message can be generated to notify the relevant developers, as shown in the second step below.

[0071] Step 2: If the anti-counterfeiting verification result indicates that the document is forged, a document forgery prompt message will be generated.

[0072] Therefore, the authenticity of the document can be guaranteed through the aforementioned anti-counterfeiting verification steps; after the anti-counterfeiting verification of the target document is completed, the content of the document can be extracted, as shown in steps S2 to S3 below.

[0073] S2. Perform information recognition processing on the target file to obtain the content information of the target file, and use the content information as the source data of the target file. In this embodiment, if the target file is an image, for example, but not limited to, image text recognition methods such as OCR (Optical Character Recognition) can be used to recognize and extract the text in the image. If the target file is a document, the text in the document can be read directly to obtain the content information of the target file. Furthermore, after obtaining the content information (i.e., the source data) of the target file, the content information can be stored for subsequent reuse and verification.

[0074] After the identification and extraction of content information in the target file are completed, key content can be extracted from the source data so that the extracted key content can be used to form the information keyword set of the target file. Optionally, in this embodiment, preliminary information extraction is performed first, and then data error correction is performed on the preliminary extracted key fields. Then, the information keyword set of the target file can be obtained. The aforementioned preliminary information extraction process can be, but is not limited to, the steps shown in step S3 below.

[0075] S3. Using regular expression matching, information extraction processing is performed on the source data to obtain a preliminary information set of the target file. This preliminary information set contains multiple key fields. In this embodiment, for example, each key field in the preliminary information set corresponds to a file content attribute; for instance, the key field "Zhang San" corresponds to the file content attribute "name"; the key field "Han ethnicity" corresponds to the file content attribute "ethnicity," etc. In this way, the extraction of different file contents from the source data can be completed, resulting in a preliminary information set. Furthermore, in this embodiment, the use of regular expression matching to extract data from the source data is a common technique for extracting information from text paragraphs; its principle will not be elaborated further.

[0076] After obtaining the initial information set, data correction processing can be carried out, which involves secondary filtering and matching. Specifically, the data correction process can be, but is not limited to, the steps shown in S4 below.

[0077] S4. Perform data error correction processing on the preliminary information set to obtain the information keyword set of the target file after data error correction processing; In this embodiment, data error correction processing mainly includes two aspects: First, correct the key fields with the same file content attributes and retain only the key fields with the highest credibility; Second, remove the interference data in the preliminary information set; In specific applications, the error correction process can be, but is not limited to, the steps S41 to S45 below.

[0078] S41. Based on the preliminary information set, at least one field cluster to be corrected is determined, wherein any field cluster to be corrected contains multiple key fields, and the file content attributes corresponding to each key field contained in any field cluster to be corrected are the same; in specific applications, key fields in the preliminary information set that belong to the same file content attribute are divided into one category. For example, assuming that the file content attributes corresponding to the key fields "Zhang San", "Li Si", and "Wang Wu" are all names, then "Zhang San", "Li Si", and "Wang Wu" are divided into the same field cluster to be corrected; similarly, the process of dividing the remaining key fields in the preliminary information set that belong to the same file content attribute is consistent with the above example, and will not be repeated here.

[0079] After obtaining at least one cluster of fields to be corrected, the credibility score of each key field in each cluster of fields to be corrected can be performed so that the key field with the highest credibility can be used as the field to be selected. The credibility score process can be, but is not limited to, the following step S42.

[0080] S42. Obtain the credibility scoring model, and input the key fields in each field cluster to be corrected into the credibility scoring model to obtain the credibility score of each key field in each field cluster to be corrected; In specific implementation, this embodiment uses a trained SVM (Support Vector Machine) model or a random forest model as the credibility scoring model to score the credibility of the key fields in each field cluster to be corrected; Furthermore, the training process of the aforementioned credibility scoring model may be, but is not limited to: (1) performing word segmentation on the source data to obtain the word segmentation set of the target file; (2) taking each word segmentation field in the word segmentation set as input, and each word segmentation field in the target file The credibility score in the target file is output to train the initial scoring model, so that the credibility scoring model is obtained after training. In this embodiment, another word segmentation method (such as maximum matching) is used to segment the source data to obtain a word segmentation set that is different from the initial information set. Then, the word segmentation set is used as training data (of course, credibility is marked on it) to train the aforementioned SVM (Support Vector Machine) model or random forest model. Then, the predicted credibility score of each training data (i.e., model output) and the true credibility score of each training data (i.e., credibility mark) are used to adjust the model parameters until the parameters are optimal, and then the aforementioned credibility scoring model can be obtained.

[0081] Thus, after obtaining the credibility score of each key field in each cluster of fields to be corrected based on the credibility scoring model, the key fields that need to be retained in the preliminary information set can be determined, as shown in step S43 below.

[0082] S43. From each cluster of fields to be corrected, select the key field with the highest confidence score to be used as the replacement field for each cluster. In a specific application, an example is used to illustrate this step. Suppose a cluster of fields to be corrected contains three key fields: "Zhang San", "Li Si", and "Wang Wu". The confidence scores of "Zhang San", "Li Si", and "Wang Wu" are 50, 60, and 90, respectively. Then, "Wang Wu" is selected as the replacement field for this cluster. Similarly, the process of determining the replacement fields for the other clusters is the same as the example above, and will not be repeated here.

[0083] In this embodiment, after obtaining the replacement fields for each cluster of fields to be corrected, data correction processing can be performed on the preliminary information set, as shown in step S44 below.

[0084] S44. For any of the fields to be corrected, update the key fields in the preliminary information set that belong to any of the fields to be corrected to the replacement fields of that field to be corrected, so that after updating all the key fields in the preliminary information set that belong to each field to be corrected, an error correction information set is obtained; in specific applications, based on the aforementioned example, it is equivalent to replacing the key fields "Zhang San" and "Li Si" in the preliminary information with "Wang Wu", so that the key field corresponding to the name as a file content attribute is only "Wang Wu"; for example Therefore, step S44 above is equivalent to retaining only the replacement field of any one of the error-to-correct field clusters in the key fields belonging to any one of the error-to-correct field clusters in the preliminary information set. In this way, after correcting the key fields corresponding to the remaining error-to-correct field clusters in the preliminary information set according to the above method, the error correction information set can be obtained. In addition, after obtaining the error correction information set, the KNN nearest neighbor algorithm can be used to complete the data in the error correction information set to avoid data loss. Then, the interference data can be removed, as shown in step S45 below.

[0085] S45. Perform data verification processing on the error correction information set to obtain the information keyword set after data verification processing; in specific applications, the removal of interfering data can be achieved by using the length of each key field in the error correction information set, thereby completing the data verification. The aforementioned process can be, but is not limited to, as shown in steps S45a to S45c below.

[0086] S45a. For any key field in the error correction information set, determine the standard length of the key field based on the file content attribute corresponding to the key field. In this embodiment, since each key field has a specific meaning (i.e., a file content attribute), the standard length of each key field can be determined by the file content attribute corresponding to each key field. For example, if the file content attribute is a name, its standard length can be, but is not limited to, 2-4 characters; if the file content attribute is an ID card, its standard length can be, but is not limited to, 18 characters. Of course, the standard length of the key fields corresponding to other file content attributes can be specifically set according to their meaning, and will not be listed here.

[0087] After obtaining the standard length of any key field, its actual length is compared with the standard length to determine whether the key field meets the data extraction requirements, as shown in steps S45b and S45c below.

[0088] S45b. Determine whether the actual length of any of the key fields meets the standard length.

[0089] S45c. If the actual length of any key field is less than the standard length, the source data is reprocessed using regular expression matching until the actual length of any key field meets the standard length, thus obtaining the information keyword set. In this embodiment, if the actual length is less than the standard length, it is necessary to return to step S3 and reprocess the information extraction until the actual length of each key field in the obtained error correction information set meets the standard length. Similarly, if the actual length of any key field is greater than the standard length, it is necessary to determine whether the key field contains special characters, such as "*" or " / ". If so, regular expressions are used to remove the special characters in the key field. After removing all special characters from all key fields in the error correction information set, the information keyword set is obtained. In addition, in this embodiment, if the actual length of any key field still does not meet the standard length after removing the special characters, it is also necessary to return to step S3 and reprocess the information extraction.

[0090] Thus, by going through the aforementioned steps S45a to S45c, we can remove interfering characters from the content, make the corresponding data match, and thus obtain the final information keywords of the target file.

[0091] Of course, in this embodiment, as explained above, if the anti-counterfeiting verification result of the target file is that the file is counterfeit, a file counterfeiting prompt message will be generated. Thus, after obtaining the information keyword set of the target file, the information keyword set can be associated with the file counterfeiting prompt message and stored in the database so as to realize the association between the extracted information and the anti-counterfeiting prompt message. Therefore, when using the information, the corresponding anti-counterfeiting prompt message can be obtained.

[0092] Therefore, through the information extraction method described in detail in steps S1 to S4 above, this invention uses machine recognition and processing to automatically extract content from images and documents. After extraction, the extracted content is corrected to obtain a set of information keywords with high accuracy. Thus, compared with manual extraction, this method is not only more efficient but also reduces labor costs, making it suitable for large-scale application and promotion in the field of information extraction technology.

[0093] In one possible design, the second aspect of this embodiment provides an example of using the information extraction method described in the first aspect of the embodiment to perform information extraction.

[0094] Assuming the target file is an image, and that image is an academic certificate, after the anti-counterfeiting detection steps, information recognition processing can be performed. At this point, the obtained source data can be, but is not limited to, the following: {'code':0,'desc':'success',“data”:”Ministry of Education Online Verification Report of Academic Status Update Date: 2*** Year * Month * Day Name *** Gender * ID Number 5135251983****0214 Ethnicity Date of Birth **** Year ** Month ** Day Graduation Photo Level Institution Chongqing Technology and Business University Undergraduate Department Class Major Business Administration Student ID Type Regular Full-time Enrollment Time 2*** Year 0 * Month 0 * Day Study System Type Academic Status Regular Higher Education Graduation (Graduation Date: 2*** (Date) Online Verification ****** Online Verification Code Scan with WeChat or use the mini-program to verify online. 1. Scan the code to obtain the "China Higher Education Student Information Network Report Online Verification" mini-program. 2. Precautions for using the mini-program to verify: 1. The "Online Verification Report of Student Status" is the query result of the Ministry of Education's electronic registration and filing of student status. 2. Report content verification method: ① Click the online verification code in the report (electronic version) to verify online; ② Log in to the "Online Verification System" of the China Higher Education Student Information Network and enter the online verification code to verify; ③ Use the "China Higher Education Student Information Network Report Online Verification" WeChat mini-program to scan the code for verification. To prevent fake reports, please use this mini-program to scan and verify, and do not use other third-party scanning programs.

[0095] Subsequently, information extraction processing is performed on the source data to obtain a preliminary information set, and data error correction is performed on the preliminary information set; finally, the obtained information keyword set is: {'code':0,'desc':'success','data':{'title':'Online Verification Report of Student Status under the Ministry of Education','name':'***','id_card':'5135251983****0214','org_code':*****,'date_information':'2022-**-** 09:39:38','gps_information':”,'sex':'male','nation':”,'birth':'August 4, 19**','graduation':'Central ** University','education':'undergraduate','faculty':”,'class':”,'major':'Computer Science and Technology','stu_number':”,'form':'Full-time General','admission_time':'* month, 20**','system':”,'type':'General Higher Education','stu_status':'Graduated (Graduation Date: ** month, 2***), 'edu':0, 'images':"['img_1670549978311_1.png', 'img_1670549978311_2.png']", 'bizNo':'XYD653530544439492608'}}.

[0096] It can be seen from this that this method can accurately extract the key content in the picture, and it is very suitable for the field of information extraction technology.

[0097] Furthermore, the present invention can also be applied to static text data parsing and daily maintenance control, and integrates technologies such as data security encryption, data security, data permissions, background control, extraction, and integration, and accesses various development languages to quickly implement the extraction of content in documents and images.

[0098] As Figure 2 shown, the third aspect of this embodiment provides a hardware device for implementing the information extraction method described in the first aspect of the embodiment, including:

[0099] A data acquisition unit for acquiring a target file, where the target file includes documents and pictures.

[0100] An information recognition unit for performing information recognition processing on the target file to obtain the content information of the target file and using the content information as the source data of the target file.

[0101] The information extraction unit is used to perform information extraction processing on the source data using a regular expression matching method to obtain a preliminary information set of the target file, wherein the preliminary information set contains multiple key fields.

[0102] The information correction unit is used to perform data correction processing on the preliminary information set so as to obtain the information keyword set of the target file after the data correction processing.

[0103] The working process, working details and technical effects of the device provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0104] like Figure 3 As shown, the fourth aspect of this embodiment provides another commission prediction device for employment tasks. Taking the device as an electronic device as an example, it includes: a memory, a processor, and a transceiver that are connected in sequence. The memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the information extraction method as described in the first aspect of the embodiment.

[0105] For specific examples, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory, etc.; specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0106] In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. For example, the processor may not be limited to microprocessors of the STM32F105 series, reduced instruction set computer (RISC) microprocessors, x86 architecture processors, or processors with integrated neural network processing units (NPUs). The transceiver may be, but is not limited to, a Wi-Fi transceiver, a Bluetooth transceiver, a General Packet Radio Service (GPRS) transceiver, a ZigBee (a low-power LAN protocol based on the IEEE 802.15.4 standard) transceiver, a 3G transceiver, a 4G transceiver, and / or a 5G transceiver. Furthermore, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0107] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0108] The fifth aspect of this embodiment provides a storage medium that stores instructions containing the information extraction method described in the first aspect of the embodiment. That is, the storage medium stores instructions that, when executed on a computer, perform the information extraction method as described in the first aspect of the embodiment.

[0109] The storage medium refers to a carrier for storing data, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0110] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0111] The sixth aspect of this embodiment provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the information extraction method as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0112] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An information extraction method, characterized in that, include: Obtain the target file, wherein the target file includes documents or images; The target file is subjected to information recognition processing to obtain the content information of the target file, and the content information is used as the source data of the target file. Using regular expression matching, information extraction processing is performed on the source data to obtain a preliminary information set of the target file, wherein the preliminary information set contains multiple key fields; The preliminary information set is subjected to data error correction processing to obtain the information keyword set of the target file after the data error correction processing; Each key field in the preliminary information set corresponds to a file content attribute. The preliminary information set undergoes data correction processing to obtain the information keyword set of the target file, including: Based on the preliminary information set, at least one field cluster to be corrected is identified, wherein any field cluster to be corrected contains multiple key fields, and the file content attributes corresponding to each key field contained in any field cluster to be corrected are the same. Obtain the credibility scoring model, and input the key fields in each field cluster to be corrected into the credibility scoring model to obtain the credibility score of each key field in each field cluster to be corrected. From each cluster of fields to be corrected, the key fields with the highest confidence scores are selected as replacement fields for each cluster of fields to be corrected. For any of the fields to be corrected, the key fields in the preliminary information set that belong to any of the fields to be corrected are updated to the replacement fields of any of the fields to be corrected, so that after all the key fields in the preliminary information set that belong to each of the fields to be corrected are updated, the error correction information set is obtained. The error correction information set is subjected to data verification processing to obtain the information keyword set after data verification processing.

2. The method according to claim 1, characterized in that, The method further includes: The source data is segmented to obtain the segmentation set of the target file; Using each segmentation field in the segmentation set as input and the credibility score of each segmentation field in the target file as output, an initial scoring model is trained to obtain the credibility scoring model after training. The initial scoring model includes an SVM model or a random forest model.

3. The method according to claim 1, characterized in that, The error correction information set is subjected to data verification processing to obtain the information keyword set after data verification processing, including: For any key field in the error correction information set, the standard length of the key field is determined based on the file content attribute corresponding to the key field. Determine whether the actual length of any of the key fields meets the standard length. If the actual length of any of the key fields is less than the standard length, then the regular expression matching method is used again to extract information from the source data until the actual length of any of the key fields meets the standard length, thus obtaining the information keyword set.

4. The method according to claim 3, characterized in that, If the actual length of any of the key fields is greater than the standard length, the method further includes: Determine whether any of the key fields contains special characters; If so, then use regular expressions to remove special characters from any of the key fields, so that after all special characters in all key fields within the error correction information set have been removed, the information keyword set is obtained.

5. The method according to claim 1, characterized in that, Before performing information recognition processing on the target file, the method further includes: The target file is subjected to anti-counterfeiting verification processing to obtain the anti-counterfeiting verification result; If the anti-counterfeiting verification result indicates that the document is forged, a document forged prompt message will be generated; Accordingly, after obtaining the set of information keywords for the target file, the method further includes: The set of information keywords is associated with the document forgery warning information and stored in the database.

6. The method according to claim 5, characterized in that, The target file is subjected to anti-counterfeiting verification processing to obtain the anti-counterfeiting verification result, including: If the target file is an image, then the first file attribute information of the target file is read, and it is determined whether there is sensitive information in the first file attribute information; If so, the NRSS detection algorithm is used to perform anti-counterfeiting detection on the target file, and the anti-counterfeiting detection score of the target file is obtained; Determine whether the anti-counterfeiting detection score is greater than a preset threshold; If so, the anti-counterfeiting verification result is generated as a document forgery; If the target file is a document, then the second file attribute information of the target file is read, wherein the second file attribute information includes the creation time and creation location; Determine whether the second file attribute information is abnormal; If so, the anti-counterfeiting verification result is generated as a document forgery.

7. An information extraction device, characterized in that, An apparatus for performing the information extraction method according to any one of claims 1 to 6, wherein the apparatus comprises: A data acquisition unit is used to acquire a target file, wherein the target file includes a document or an image; An information recognition unit is used to perform information recognition processing on the target file to obtain the content information of the target file, and use the content information as the source data of the target file; The information extraction unit is used to perform information extraction processing on the source data using a regular expression matching method to obtain a preliminary information set of the target file, wherein the preliminary information set contains multiple key fields; The information correction unit is used to perform data correction processing on the preliminary information set so as to obtain the information keyword set of the target file after the data correction processing.

8. An electronic device, characterized in that, include: A memory, a processor, and a transceiver are sequentially connected in communication, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the information extraction method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer, perform the information extraction method as described in any one of claims 1 to 6.

Images