Data Processing Method, Device, and Computer-Readable Storage Medium

Abstract

The present invention discloses a data processing method, device and computer-readable storage medium. The method includes: in the HTML elements of an HTML document, determining the table header HTML elements corresponding to each table header name and the table text HTML elements corresponding to each table text in the bank statement document; based on the table header HTML elements and the table text HTML elements, determining the position information of each table header name and each table text in the bank statement document; traversing each table text HTML element based on the order of the table text HTML elements in the HTML document, and generating a key-value pair corresponding to each currently traversed table text HTML element based on the position information; determining the statement record information corresponding to the bank statement document based on the key-value pair, and outputting the statement record information when the traversal of each table text HTML element is completed. The present invention transforms bank statement information into structured statement data through document parsing, ensuring the accuracy and effectiveness of the parsing result of bank statement information.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular, to a data processing method, device, and computer-readable storage medium. Background Art

[0002] A bank statement refers to the transaction details of a bank account. In modern society, bank statements play an increasingly important role. When people apply for housing loans, car loans, credit cards, etc., they need to provide bank statements. Classifying and summarizing various incomes and expenditures in bank statements can comprehensively analyze and evaluate the financial strength and debt repayment ability of the account holder. However, due to the inconsistency of systems among different banks, there are significant differences between the PDF documents of bank statements obtained from different banks. In order to integrate the bank statement information of different banks into the same type of information, a unified method for parsing the bank statement documents of multiple mainstream banks needs to be sought.

[0003] In related technologies, in order to identify and parse bank statement documents, OCR technology is generally used to quickly scan the statement documents and obtain relevant information. However, due to the influence of factors such as image brightness and darkness, font size, and color differences, the accuracy of the parsed bank statement information is not high. Summary of the Invention

[0004] The main purpose of the present invention is to provide a data processing method, device, and computer-readable storage medium, aiming to solve the technical problem of low accuracy of the parsing result of bank statement information.

[0005] To achieve the above object, the present invention provides a data processing method, and the data processing method includes the following steps:

[0006] Convert a bank statement document into an HTML document, and in the HTML elements of the HTML document, determine the header HTML elements corresponding to each header name and the table text HTML elements corresponding to each table text in the bank statement document;

[0007] Based on the header HTML elements and the table text HTML elements, determine the position information of each header name and each table text in the bank statement document;

[0008] Based on the order of each table text HTML element in the HTML document, traverse each table text HTML element, and generate a key-value pair corresponding to each currently traversed table text HTML element based on the position information, where the value data of the key-value pair is the current table text corresponding to the currently traversed table text HTML element, and the key data of the key-value pair is the header name corresponding to the current table text;

[0009] Determine the transaction record information corresponding to the bank statement document based on the key-value pairs, and output the transaction record information when the traversal of each of the table text HTML elements is completed.

[0010] Preferably, the step of traversing each of the table text HTML elements based on the order of the table text HTML elements in the HTML document and generating key-value pairs corresponding to each currently traversed table text HTML element based on the position information includes:

[0011] Use the table text corresponding to the currently traversed table text HTML element as the to-be-processed table text, and based on the position information, obtain the table range corresponding to the to-be-processed table text, where the table range is the abscissa range of the to-be-processed table text, and the position information includes the starting coordinates of the table headers corresponding to each of the table header names, the pixel widths of the table headers, and the starting coordinates of the table texts corresponding to each of the table texts, the pixel widths of the table texts;

[0012] Based on the position information and a first preset range coefficient, obtain the table header ranges corresponding to each of the table header names, where the table header range is the abscissa range corresponding to each of the table header names;

[0013] Compare each of the table header ranges based on the table range, and determine whether there is a target table header range that covers the table range among each of the table header ranges;

[0014] If there is, generate the key-value pair based on the to-be-processed table text and the table header name corresponding to the target table header range.

[0015] Preferably, after the step of comparing each of the table header ranges based on the table range and determining whether there is a target table header range that covers the table range among each of the table header ranges, it further includes:

[0016] If not, update the acquisition times of the table header range, and determine a second preset range coefficient based on the acquisition times and a preset ratio;

[0017] Use the second preset range coefficient as the first preset range coefficient, and return to execute the step of obtaining the table header ranges corresponding to each of the table header names based on the position information and the first preset range coefficient.

[0018] Preferably, the step of using the table text corresponding to the currently traversed table text HTML element as the to-be-processed table text and obtaining the table range corresponding to the to-be-processed table text based on the position information includes:

[0019] Based on the table text HTML element, obtain the starting coordinates of the table text corresponding to the to-be-processed table text and the pixel width of the table text;

[0020] Based on the starting coordinates of the table text and the pixel width of the table text, determine the table text range.

[0021] Preferably, the step of obtaining the table header range corresponding to each table header name based on the position information and the first preset range coefficient includes:

[0022] Based on the table header HTML element, obtain the starting coordinates of each table header name and the pixel width of the table header;

[0023] Based on the starting coordinates of the table header and the pixel width of the table header, determine the central position information corresponding to each table header name;

[0024] Based on the central position information and the first preset range coefficient, determine the table header range.

[0025] Preferably, the step of determining the transaction record information corresponding to the bank statement document based on the key-value pair and outputting the transaction record information when traversing each table text HTML element includes:

[0026] If the key-value pair is not the first generated key-value pair, then use the key-value pair as the first key-value pair, use the previous key-value pair adjacent to the first key-value pair as the second key-value pair, and match the first table header name corresponding to the first key-value pair with the second table header name corresponding to the second key-value pair;

[0027] If the serial number corresponding to the first table header name is 1 and the first table header name is different from the second table header name, then create a new first transaction record and store the first key-value pair in the first transaction record, and update the transaction record information based on the first transaction record, or;

[0028] If the serial number corresponding to the first table header name is not 1 and the first table header name is different from the second table header name, then store the first key-value pair in the second transaction record containing the second key-value pair, and update the transaction record information based on the second transaction record, or;

[0029] If the first table header name is the same as the second table header name, then update the second table text corresponding to the second key-value pair based on the first table text corresponding to the first key-value pair, and update the transaction record information based on the updated second table text;

[0030] When traversing each table text HTML element is completed, output the transaction record information.

[0031] Preferably, before the step of, if the key-value pair is not the first generated key-value pair, taking the key-value pair as the first key-value pair, taking the previous key-value pair adjacent to the first key-value pair as the second key-value pair, and matching the first table header name corresponding to the first key-value pair with the second table header name corresponding to the second key-value pair, the method further includes:

[0032] If the key-value pair is the first generated key-value pair, create a first flow record and store the key-value pair in the first flow record;

[0033] Update the flow record information based on the first flow record.

[0034] Preferably, the step of converting the bank statement document into an HTML document and, in the HTML elements of the HTML document, determining the table header HTML elements corresponding to the respective table header names in the bank statement document and the table text HTML elements corresponding to the respective table text is as follows:

[0035] Convert the bank statement document into an HTML document, and obtain the HTML elements based on the HTML document;

[0036] Based on the HTML elements and the preset feature information of each preset bank, determine the bank corresponding to the bank statement document;

[0037] Based on the preset feature information corresponding to the bank, determine the table header HTML elements and the table text HTML elements.

[0038] In addition, to achieve the above object, the present invention further provides a data processing device, including: a memory, a processor, and a data processing program stored on the memory and executable on the processor, where when the data processing program is executed by the processor, the steps of the data processing method as described above are implemented.

[0039] In addition, to achieve the above object, the present invention further provides a computer-readable storage medium, on which a data processing program is stored, and when the data processing program is executed by a processor, the steps of the data processing method as described above are implemented.

[0040] The present invention converts a bank statement document into an HTML document, and in the HTML elements of the HTML document, determines the header HTML elements corresponding to each header name in the bank statement document and the table text HTML elements corresponding to each table text; based on the header HTML elements and the table text HTML elements, determines the position information of each header name and each table text in the bank statement document; based on the order of the table text HTML elements in the HTML document, traverses each table text HTML element, and generates a key-value pair corresponding to each currently traversed table text HTML element based on the position information, where the value data of the key-value pair is the current table text corresponding to the currently traversed table text HTML element, and the key data of the key-value pair is the header name corresponding to the current table text; determines the statement record information corresponding to the bank statement document based on the key-value pair, and outputs the statement record information when the traversal of each table text HTML element is completed. First, convert the bank statement PDF document into an HTML document, then read the content of the document and parse it, and finally transform the bank statement information in the bank statement PDF document into structured statement data and output it, so that accurate and unified statement data can be obtained according to different bank statement documents, and the accuracy and effectiveness of the parsing result of the bank statement information are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic structural diagram of a data processing device in the hardware operating environment related to the solution of the embodiment of the present invention;

[0042] Figure 2 is a schematic flowchart of the first embodiment of the data processing method of the present invention;

[0043] Figure 3 is a schematic diagram of the start flag and end flag of the statement in an embodiment of the data processing method of the present invention;

[0044] Figure 4 is a schematic diagram of the HTML elements of the bank statement information in an embodiment of the data processing method of the present invention.

[0045] The implementation, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] As Figure 1 shown, Figure 1 is a schematic structural diagram of a data processing device in the hardware operating environment related to the solution of the embodiment of the present invention.

[0048] In an embodiment of the present invention, the data processing device may be a PC, or may be a terminal device with a display function such as a smart phone, a tablet computer, a portable computer, etc.

[0049] As Figure 1 shown, the data processing device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or may be a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the foregoing processor 1001.

[0050] Optionally, the data processing device may further include a camera, an RF (Radio Frequency) circuit, sensors, an audio circuit, a WiFi module, etc. Among them, the sensors such as a light sensor, a motion sensor, and other sensors will not be elaborated herein.

[0051] Those skilled in the art can understand that Figure 1 the terminal structure shown in

[0052] As Figure 1 shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data processing program.

[0053] In Figure 1 the data processing device shown, the network interface 1004 is mainly used to connect to a background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to a client (user side) and perform data communication with the client; and the processor 1001 may be used to call the data processing program stored in the memory 1005.

[0054] In this embodiment, the data processing device includes: a memory 1005, a processor 1001, and a data processing program stored on the memory 1005 and executable on the processor 1001. When the processor 1001 calls the data processing program stored in the memory 1005, it executes the steps of the data processing method in each of the following embodiments.

[0055] The present invention also provides a method. Referring to Figure 2 , Figure 2 is a schematic flowchart of the first embodiment of the data processing method of the present invention.

[0056] In this embodiment, the data processing method includes the following steps:

[0057] Step S101: Convert the bank statement document into an HTML document, and in the HTML elements of the HTML document, determine the table header HTML elements corresponding to each table header name and the table body HTML elements corresponding to each table body text in the bank statement document;

[0058] In this embodiment, after uploading the bank statement document corresponding to the bank statement information, convert the bank statement document into an HTML (HyperText Mark-up Language) document. Then, various HTML elements corresponding to the bank statement information can be obtained from the HTML document. For example, if the bank statement document is a PDF document, convert the PDF document into an HTML document, and identify the content of each HTML element of the HTML document. According to the recognition result, determine the bank corresponding to the PDF document. Then, read the table header name, the start flag of the statement, and the end flag of the statement of this bank from the database. Then, in the HTML document, according to the table header name, the start flag of the statement, and the end flag of the statement, determine the table header HTML elements corresponding to each table header name in the HTML elements, and the table body HTML elements corresponding to each table body text between the start flag of the statement and the end flag of the statement. Among them, the table in the bank statement document is divided into a table header part and a table body part. The table header name is each table header name included in the table header part of this bank, and each table header name corresponds to a serial number in sequence. The start flag of the statement is the previous text information of the table body part, generally the last table header name. The end flag of the statement is the next text information of the table body part, as Figure 3 shown.

[0059] In another embodiment, the bank statement document is not necessarily a PDF document and may be any document containing bank statement information. For example, an EXCEL document or a WORD document.

[0060] In another embodiment, the header names, the start flag of the bank statement, and the end flag of the bank statement corresponding to each bank can be obtained in advance, and the corresponding bank marks can be added, and then stored in the database as the preset feature information of each bank, so as to perform feature information recognition, thereby determining the bank corresponding to the bank statement document, and determining the header HTML elements and the table text HTML elements in the HTML document.

[0061] Step S102: Based on the header HTML elements and the table text HTML elements, determine the position information of each of the header names and each of the table text in the bank statement document;

[0062] In this embodiment, in the HTML document converted from the bank statement document, after determining the header HTML elements and the table text HTML elements in each HTML element, the position information corresponding to each header name can be determined according to the header HTML elements, and the position information corresponding to each table text can be determined according to the table text HTML elements, where the position information includes the header start coordinates and the header pixel width of each header name, and the table text start coordinates and the table text pixel width of each table text.

[0063] Specifically, an HTML element includes an element tag and element content. The header name corresponding to each header HTML element is the <text>< / text> element content between the tags corresponding to each header HTML element, and the table text corresponding to each table text HTML element is the <text>< / text> element content between the tags corresponding to each table text HTML element. According to the canvas element in the HTML element and the <text>The font-family property and font-size property in the label can determine the pixel width of the table header corresponding to each table header name and the starting coordinates of the table header, which are used as the position information of each table header name. Similarly, the table text corresponding to each table text HTML element is the <text>< / text> element content between the labels. According to the canvas element in the HTML element and the <text>The font-family property and font-size property in the label can determine the corresponding table text pixel width and the starting coordinates of the table text for each piece of table text, serving as the position information for each piece of table text, such as Figure 4 shown

[0064] It should be noted that for a certain text content, the starting coordinate is the coordinate of the first pixel point corresponding to the text content in the horizontal coordinate direction, and the pixel width is the sum of the widths of all pixel points included in the text content in the horizontal coordinate direction.

[0065] Step S103: Traverse each of the table text HTML elements based on the order of the table text HTML elements in the HTML document, and generate a key-value pair corresponding to each currently traversed table text HTML element based on the position information, where the value data of the key-value pair is the current table text corresponding to the currently traversed table text HTML element, and the key data of the key-value pair is the header name corresponding to the current table text;

[0066] In this embodiment, each table text HTML element is traversed in the order of the table text HTML elements in the HTML document. According to the currently traversed table text HTML element, the position information of the table text corresponding to the HTML element is obtained, and the position information of each header name is obtained to generate a key-value pair corresponding to the table text, where the value data of the key-value pair is the current table text corresponding to the currently traversed table text HTML element, the key data of the key-value pair is the header name corresponding to the current table text, and the key-value pair is a string formed by the key data and the value data.

[0067] Specifically, the table text corresponding to the currently traversed table text HTML element is used as the table text to be processed, and according to the corresponding <text>Label, obtain the starting coordinates of the form text and the pixel width of the form text to be processed. According to the starting coordinates of the form text and the pixel width of the form text, the horizontal coordinate range of the form text to be processed can be determined as the form range. At the same time, according to the <text>Labels are used to determine the starting coordinates and pixel widths of the table headers for each header name. Then, based on the starting coordinates and pixel widths of the table headers for each header name, the abscissa of the center position of each header name is determined. Subsequently, based on the abscissa of the center position and a preset range coefficient, the abscissa range corresponding to each header name is determined as the header range. Based on the table text range and each header range, key-value pairs are generated. For example, when the table text range is covered by a certain header range, that is, the minimum boundary of the table text range is greater than the minimum boundary of the header range, and the maximum boundary of the table text range is less than the maximum boundary of the header range, a key-value pair is generated according to the table text to be processed and the header name corresponding to the header range. If the table text range is not covered by any header range, the preset range coefficient is re-determined according to a preset ratio, and the header ranges corresponding to each header name are re-determined based on the new preset range coefficient and the abscissa of the center position of each header name. Then, it is determined again whether the table text range is covered by the header range corresponding to a certain header name. If it is covered, the corresponding key-value pair is generated. If it is not covered, the above steps of re-determining the preset range coefficient are repeated until a header range that can cover the table text range is generated, and the key-value pairs corresponding to the table text to be processed are generated.

[0068] In another embodiment, the abscissa of the center position of the table text to be processed can be obtained, and the abscissa of the center position of each header name can be obtained. Since in actual bank statements, the center positions of the header names and the corresponding table text are closest in the abscissa direction, therefore, among the abscissas of the center positions corresponding to each header name, the abscissa with the smallest difference from the abscissa of the center position of the table text to be processed is determined, and a key-value pair is determined according to the header name corresponding to this abscissa and the table text to be processed.

[0069] In another embodiment, the preset range coefficient can be reduced to narrow the header range and compare it with the table text range until a header range covered by the table text range is determined among each header name, and a key-value pair is generated according to the header name corresponding to this header range and the table text to be processed; it is also possible to determine the header ranges of each header name based on the starting coordinates and pixel widths of the headers, and then determine the abscissa of the center position of the table text to be processed based on the starting coordinates and pixel widths of the table text. According to the range coefficient and the abscissa of the center position of the table text to be processed, the table text range of the table text to be processed is determined. Then, as the range coefficient increases, the table text range increases, and key-value pairs are determined according to the header ranges covered by the table text range, or as the range coefficient decreases, the table text range shrinks, and key-value pairs are determined according to the header ranges covering the table text range.

[0070] In another embodiment, the ratio of the number of table text HTML elements to the number of table header HTML elements can be determined, and this ratio corresponds to the number of lines of table text in the bank statement information. According to the order of each table text HTML element and each table header HTML element, the table text HTML elements can be sequentially and one-to-one corresponded with the table header HTML elements to generate key-value pairs. When corresponding to the last table header HTML element and generating the corresponding key-value pair, then start corresponding to the next table text HTML element from the first table header HTML element until the last table text HTML element generates the corresponding key-value pair.

[0071] Step S104, determine the transaction record information corresponding to the bank statement document based on the key-value pairs, and output the transaction record information when the traversal of each of the table text HTML elements is completed.

[0072] In this embodiment, when performing the above traversal process on each table text HTML element, each time a key-value pair is generated, the transaction record information corresponding to the bank statement document is updated according to the generated key-value pair. When the traversal is completed, that is, after updating the transaction record information according to the key-value pair corresponding to the last table text HTML element, the transaction record information is output. For example, when generating the key-value pair corresponding to the table text to be processed, if the table text to be processed is the table text corresponding to the first table text HTML element in the HTML document, that is, this key-value pair is the first key-value pair generated in the bank statement document, then a new transaction record is created and this key-value pair is stored in this transaction record; if the generated key-value pair is not the first generated key-value pair, then it is compared with the adjacent previous key-value pair, so as to update the transaction record information according to the comparison result. For example, each table header name corresponds to a serial number, and this serial number corresponds to the order of the table header HTML element corresponding to each table header name in the HTML document. If the serial number corresponding to the table header name corresponding to the generated key-value pair is 1, and this table header name is different from the table header name corresponding to the previous key-value pair, then a new transaction record is created and the generated key-value pair is stored in this transaction record; if the serial number corresponding to the table header name corresponding to the generated key-value pair is not 1, and this table header name is different from the table header name corresponding to the previous key-value pair, then the generated key-value pair is stored in the transaction record where the previous key-value pair is located; if the table header name corresponding to the generated key-value pair is the same as the table header name corresponding to the previous key-value pair, then directly append the table text corresponding to the generated key-value pair after the table text corresponding to the previous key-value pair. Finally, when the traversal is completed, that is, after each generated key-value pair is updated to the transaction record information according to the above steps, the transaction record information is output.

[0073] In another embodiment, when traversing each table text HTML element, the flow record information may not be updated. When the traversal is completed, all generated key-value pairs are obtained, and each key-value pair updates the flow record information in the above steps in sequence according to the order of the corresponding table text HTML element.

[0074] The data processing method proposed in this embodiment converts a bank statement PDF document into an HTML document, and in the HTML elements of the HTML document, determines the table header HTML elements corresponding to each table header name and the table text HTML elements corresponding to each table text in the bank statement document; and based on the table header HTML elements and the table text HTML elements, determines the position information of each table header name and each table text in the bank statement document; then, based on the order of each table text HTML element in the HTML document, traverses each table text HTML element, and generates a key-value pair corresponding to the currently traversed table text HTML element based on the position information, where the value data of the key-value pair is the current table text corresponding to the currently traversed table text HTML element, and the key data of the key-value pair is the table header name corresponding to the current table text; thereby determining the flow record information corresponding to the bank statement document based on the key-value pair, and outputting the flow record information when the traversal of each table text HTML element is completed. By converting the bank statement PDF document into an HTML document, then reading and parsing the content of the document, generating corresponding key-value pairs according to the content and determining the flow record information according to the key-value pairs, finally transforming the bank statement information in the bank statement PDF document into structured flow data and outputting it as the flow record information, so that accurate and unified flow data can be obtained according to different bank statement documents, and the accuracy and effectiveness of the parsing result of the bank statement information are ensured.

[0075] Based on the first embodiment, a second embodiment of the data processing method of the present invention is proposed. In this embodiment, step S103 includes:

[0076] Step S201, taking the table text corresponding to the currently traversed table text HTML element as the to-be-processed table text, and based on the position information, obtaining the table range corresponding to the to-be-processed table text, where the table range is the horizontal coordinate range of the to-be-processed table text, and the position information includes the table header start coordinates, table header pixel widths corresponding to each table header name, and the table start coordinates, table text pixel widths corresponding to each table text;

[0077] Step S202, based on the position information and the first preset range coefficient, obtaining the table header range corresponding to each table header name, where the table header range is the horizontal coordinate range corresponding to each table header name;

[0078] Step S203: Compare each of the header ranges based on the table text range, and in each of the header ranges, determine whether there is a target header range that covers the table text range.

[0079] Step S204: If there is such a target header range, generate the key-value pair based on the table text to be processed and the header name corresponding to the target header range.

[0080] After step S203, it further includes:

[0081] Step S301: If there is no such target header range, update the acquisition times of the header range, and determine a second preset range coefficient based on the acquisition times and a preset ratio.

[0082] Step S302: Use the second preset range coefficient as the first preset range coefficient, and return to execute the step of obtaining the header range corresponding to each header name based on the position information and the first preset range coefficient.

[0083] In this embodiment, when traversing each table text HTML element in the order of the table text HTML elements in the HTML document, use the table text corresponding to the currently traversed table text HTML element as the table text to be processed, obtain the position information of the table text to be processed according to the table text HTML element, determine the horizontal coordinate range of the table text to be processed based on the position information of the table text to be processed as the table text range, then obtain the position information of each header name, and determine the horizontal coordinate range corresponding to each header name as the header range corresponding to each header name according to the position information of each header name and the preset range coefficient, compare the table text range with the header ranges corresponding to each header name, so as to determine the header name corresponding to the table text to be processed according to the comparison result and generate a key-value pair. Among them, the key-value pair is a string formed by key data and value data. The value data of the key-value pair is the table text to be processed, and the key data of the key-value pair is the header name corresponding to the table text to be processed. For example, use the table text corresponding to the currently traversed table text HTML element as the table text to be processed, and according to the canvas element in the HTML element and the <text>The font-family property and font-size property in the label determine the text pixel width and the starting coordinates of the form text corresponding to the form text to be processed, and serve as the position information of the form text to be processed. Similarly, according to the canvas element in the HTML element and the <text>The font-family attribute and font-size attribute in the tag determine the header pixel width and header starting coordinates corresponding to each header name as the position information of each header name. Suppose the starting coordinates of the table text to be processed are (x, y), the pixel width of the table text is w, and the starting coordinates of the header names are (x, y). i ,y i ), the header pixel width is d i , i is the serial number of each header name. According to the starting coordinates of the table text and the pixel width of the table text, the horizontal coordinate range of the table text to be processed can be determined, that is, the table text range is [x, x+w]. According to the starting coordinates of each header name and the pixel width of the header name, the horizontal coordinate x of the center position of each header name can be calculated. mi , the formula is: mi =x i +d i / 2, and then set the range coefficient λ, the value of λ is: λ=0.05k, k=1,2,3,..., k is the preset ratio, according to the range coefficient λ and the horizontal coordinate x of the center position mi , we can start from k=1 and determine the horizontal axis range corresponding to each header name, that is, the header range is [x mi -λx mi , x mi +λx mi ], if the table range [x, x+w] is replaced by the table header range [x mi -λx mi , x mi +λx mi ], that is, the minimum boundary of the table text range is greater than the minimum boundary of the table header range, and the maximum boundary of the table text range is less than the maximum boundary of the table header range, then a key-value pair key:value is generated, where key represents the table header name corresponding to i, and value represents the table text to be processed; if the table text range [x, x+w] is not covered by the table header range [x mi -λx mi , x mi +λx mi ], then increase the value of k, so that λ increases, thus making the header range [x mi -λx mi , x mi +λx mi ] increases until the text range [x, x+w] is filled by the header range [x mi -λx mi , x mi +λx mi Cover and generate key-value pairs key:value corresponding to the text of the form to be processed, where the key data represents the form header name corresponding to i, and the value data represents the text of the form to be processed.

[0084] In the data processing method proposed in this embodiment, by using the text of the form corresponding to the currently traversed form HTML element as the text of the form to be processed, and based on the position information, obtain the form range corresponding to the text of the form to be processed, where the form range is the abscissa range of the text of the form to be processed, and the position information includes the starting coordinates of the form headers corresponding to each form header name, the pixel width of the form headers, and the starting coordinates of the form texts corresponding to each form text, and the pixel width of the form texts; then based on the position information and the first preset range coefficient, obtain the form header range corresponding to each form header name, where the form header range is the abscissa range corresponding to each form header name; then compare each form header range based on the form range, and determine whether there is a target form header range that covers the form range among each form header range; if so, generate the key-value pair based on the text of the form to be processed and the form header name corresponding to the target form header range; if not, update the number of times of obtaining the form header range, and determine the second preset range coefficient based on the number of times of obtaining and the preset ratio; then use the second preset range coefficient as the first preset range coefficient, and return to execute the step of obtaining the form header range corresponding to each form header name based on the position information and the first preset range coefficient. By sequentially processing each form text in order, determine the form header name corresponding to the text of the form to be processed among each form header name according to whether the form header range covers the form range, and generate key-value pairs. If not covered, increase the form header range and repeat the previous step of determining whether the form header range covers the form range, ensuring the accuracy of the generated key-value pairs, which is beneficial to subsequently converting bank statement information into unified structured statement data and outputting it as statement record information, thereby ensuring the accuracy and effectiveness of the bank statement information parsing result.

[0085] Based on the second embodiment, a third embodiment of the data processing method of the present invention is proposed. In this embodiment, step S201 includes:

[0086] Step S401, based on the form HTML element, obtain the starting coordinates and pixel width of the form corresponding to the text of the form to be processed;

[0087] Step S402, based on the starting coordinates of the form and the pixel width of the form, determine the form range.

[0088] In this embodiment, among each form text HTML element, determine the form text HTML element corresponding to the form text to be processed. According to the form text HTML element corresponding to the form text to be processed, determine the starting coordinate and pixel width of the form text of the form text to be processed, which are used as the position information of the form text to be processed. And according to the starting coordinate and pixel width of the form text of the form text to be processed, determine the horizontal coordinate range of the form text to be processed, that is, the form text range. For example, according to the canvas element in the HTML element and the <text>The font-family property and font-size property in the label determine the starting coordinates (x, y) of the table text to be processed and the pixel width w of the table text. Then, the horizontal coordinate range of the table text to be processed in the horizontal direction can be determined as [x, x + w].

[0089] The data processing method proposed in this embodiment obtains the starting coordinates of the table text corresponding to the table text to be processed and the pixel width of the table text based on the table text HTML element, and then determines the table range based on the starting coordinates of the table text and the pixel width of the table text. The length range between the first pixel point and the last pixel point of the table text to be processed in the horizontal direction is accurately obtained as the table range. By ensuring the accuracy of the table range, the accuracy of the subsequent generated key-value pairs is ensured, thereby ensuring the accuracy and effectiveness of the final bank statement information parsing result.

[0090] Based on the second embodiment, a fourth embodiment of the data processing method of the present invention is proposed. In this embodiment, step S202 includes:

[0091] Step S501: Based on the table header HTML element, obtain the starting coordinates of each table header name and the pixel width of the table header;

[0092] Step S502: Based on the starting coordinates of the table header and the pixel width of the table header, determine the center position information corresponding to each table header name;

[0093] Step S503: Based on the center position information and the first preset range coefficient, determine the table header range.

[0094] In this embodiment, according to each table header HTML element, the starting coordinates of each table header name and the pixel width of the table header are determined as the position information of each table header name, and according to the starting coordinates of each table header name and the pixel width of the table header, the horizontal coordinate range corresponding to each table header name, that is, the table header range, is determined.

[0095] Specifically, according to the canvas element in the HTML element and each table header name corresponding to <text>The font-family property and font-size property in the label determine the starting coordinates of each table header name and the pixel width of the table header. For example, the table header names included in the bank statement information are, in order, serial number, transaction time, transaction amount, balance, summary, and counterparty information. Among them, each table header name corresponds to a serial number. In the bank statement information, sort the table header names in order from left to right, and / or in the HTML element, sort the table header HTML elements from top to bottom to obtain the serial numbers of each table header name, which are 1, 2, 3,... respectively. Then, it can be determined that the serial number of the serial number is 1, the serial number of the transaction time is 2, the serial number of the transaction amount is 3, the serial number of the balance is 4, the serial number of the summary is 5, and the serial number of the counterparty information is 6. Let i be the serial number of each table header name, and the starting coordinates of the table header be (x i , y i ). According to the <text>Labels, it is determined that the starting coordinates of the table headers corresponding to each table header name in the bank statement information are: Serial number: (x1, y1), Transaction time: (x2, y2), Transaction amount: (x3, y3), Balance: (x4, y4), Summary: (x5, y5), Transaction counterparty information: (x6, y6); Let the pixel width corresponding to the table header name be d i , then the pixel widths of the table headers corresponding to each table header name are: Serial number: d1, Transaction time: d2, Transaction amount: d3, Balance: d4, Summary: d5, Transaction counterparty information: d6. According to the starting coordinates of the table headers and the pixel widths of the table headers, the abscissa x of the center position of each table header name can be calculated mi , and the formula is: x mi =x i +d i / 2; In addition, a range coefficient λ is set, and the value of λ is: λ = 0.05k, k = 1, 2, 3,..., k is the preset ratio of the range coefficient λ, so as to determine the abscissa range corresponding to each table header name, that is, the table header range is [x mi -λx mi , x mi +λx mi .

[0096] The data processing method proposed in this embodiment obtains the starting coordinates of the table headers and the pixel widths of the table headers corresponding to each table header name based on the table header HTML elements; and determines the center position information corresponding to each table header name based on the starting coordinates of the table headers and the pixel widths of the table headers; thus, based on the center position information and the first preset range coefficient, the table header range is determined. In the above process, the preset range coefficient can change according to the change of the preset ratio, so that the table header range changes. By the changing table header range, it is ensured that each table text range has and only has one corresponding table header range. Therefore, each table text has and only has one corresponding table header name, ensuring the accuracy of the key-value pairs generated subsequently, and thus ensuring the accuracy and effectiveness of the final bank statement information parsing result.

[0097] Based on the first embodiment, a fifth embodiment of the data processing method of the present invention is proposed. In this embodiment, step S104 includes:

[0098] Step S601, if the key-value pair is not the first generated key-value pair, then use the key-value pair as the first key-value pair, use the previous key-value pair adjacent to the first key-value pair as the second key-value pair, and match the first table header name corresponding to the first key-value pair with the second table header name corresponding to the second key-value pair;

[0099] Step S602, if the serial number corresponding to the first table header name is 1 and the first table header name is different from the second table header name, create a new first flow record, store the first key-value pair in the first flow record, and update the flow record information based on the first flow record, or;

[0100] Step S603, if the serial number corresponding to the first table header name is not 1 and the first table header name is different from the second table header name, store the first key-value pair in the second flow record containing the second key-value pair, and update the flow record information based on the second flow record, or;

[0101] Step S604, if the first table header name is the same as the second table header name, update the second table text corresponding to the second key-value pair based on the first table text corresponding to the first key-value pair, and update the flow record information based on the updated second table text;

[0102] Step S605, when the traversal of each of the table text HTML elements is completed, output the flow record information.

[0103] Among them, before step S601, it further includes:

[0104] Step S701, if the key-value pair is the first generated key-value pair, create a new first flow record and store the key-value pair in the first flow record;

[0105] Step S702, update the flow record information based on the first flow record.

[0106] In this embodiment, the above traversal process is performed on each table text HTML element in the HTML document corresponding to the bank statement document. And for each traversed table text HTML element, a key-value pair key:value corresponding to the table text HTML element is generated. Wherein, the key data represents the header name corresponding to the table text HTML element, and the serial number of the header name is i. Each header name corresponds to a serial number. In the bank statement information, the header names are sorted in the order from left to right, and / or in the HTML element, the header HTML elements are sorted from top to bottom, so as to obtain the serial numbers of the header names, which are 1, 2, 3,... respectively. The value data represents the table text corresponding to the table text HTML element. For each generated key-value pair, the key-value pair is obtained, and the corresponding water record information of the bank statement document is updated according to the key-value pair. When the traversal is completed, that is, after the water record information is updated according to the key-value pair corresponding to the last table text HTML element, the water record information is output. For example, the table text corresponding to the currently traversed table text HTML element is used as the to-be-processed table text. When the key-value pair corresponding to the to-be-processed table text is generated, if the to-be-processed table text is the table text corresponding to the first table text HTML element in the HTML document, that is, the key-value pair is the first generated key-value pair in the bank statement document, a new water record record is created and the key-value pair is stored in the water record in the form of key:value; if the generated key-value pair is not the first generated key-value pair, the key-value pair is key:value, and its adjacent previous key-value pair is set as key pre :value pre , and key:value and key pre :value pre are compared. If the serial number i of the header name corresponding to the key data in key:value is 1, and the header name corresponding to the key data is different from the header name corresponding to key pre data, a new water record is created and key:value is stored in the newly created water record; if the serial number i of the header name corresponding to the key data in key:value is not 1, and the header name corresponding to the key data is different from the header name corresponding to key pre data, then key:value is stored in the water record where key pre :value pre is located; if the header name corresponding to the key data is the same as the header name corresponding to key pre data, then in the water record where key pre :value pre is located, the value pre data is updated to value pre +value, where value pre +value means that the content of the value data is concatenated to value pre After the content of the data, finally, when the key-value pairs generated by the last table text HTML element in the HTML document are updated to the flow record information according to the above steps, it is regarded that the traversal is completed, and the flow record information is output.

[0107] The data processing method proposed in this embodiment, if the key-value pair is the first generated key-value pair, a first flow record is newly created and the key-value pair is stored in the first flow record; the flow record information is updated based on the first flow record; if the key-value pair is not the first generated key-value pair, the key-value pair is used as the first key-value pair, the previous key-value pair adjacent to the first key-value pair is used as the second key-value pair, and the first table header name corresponding to the first key-value pair is matched with the second table header name corresponding to the second key-value pair; if the serial number corresponding to the first table header name is 1 and the first table header name is different from the second table header name, a first flow record is newly created and the first key-value pair is stored in the first flow record, and the flow record information is updated based on the first flow record, or; if the serial number corresponding to the first table header name is not 1 and the first table header name is different from the second table header name, the first key-value pair is stored in the second flow record containing the second key-value pair, and the flow record information is updated based on the second flow record, or; if the first table header name is the same as the second table header name, the second table text corresponding to the second key-value pair is updated based on the first table text corresponding to the first key-value pair, and the flow record information is updated based on the updated second table text; finally, when the traversal of each of the table text HTML elements is completed, the flow record information is output. A flow record is newly created according to the first obtained key-value pair to determine the start of the structured flow data. Then, after traversing each table text HTML element, each time a key-value pair is generated, it is compared with the previous adjacent key-value pair, and the flow record information is updated according to the comparison result, ensuring the accuracy and uniformity of the flow record information. Finally, the bank flow information in the bank flow PDF document is transformed into structured flow data and output as the flow record information, so that accurate and unified flow data can be obtained according to different bank flow documents, and the accuracy and effectiveness of the bank flow information parsing result are ensured.

[0108] Based on the first embodiment, a sixth embodiment of the data processing method of the present invention is proposed. In this embodiment, step S101 includes:

[0109] Step S801, converting the bank flow document into an HTML document, and obtaining the HTML element based on the HTML document;

[0110] Step S802: Based on the HTML element and the preset feature information of each preset bank, determine the bank corresponding to the bank statement document.

[0111] Step S803: Based on the preset feature information corresponding to the bank, determine the table header HTML element and the table body HTML element.

[0112] In this embodiment, after obtaining the bank statement document corresponding to the bank statement information, convert the bank statement document into an HTML (HyperText Mark-up Language) document, and then each HTML element corresponding to the bank statement information can be obtained from the HTML document.

[0113] Specifically, the bank statement document is generally a PDF document. Convert the PDF document into an HTML document, identify the feature information of the content of the HTML document, compare the feature information of the bank statement PDF document with the preset feature information of each preset bank stored in the database, determine the bank corresponding to the bank statement PDF document, and then read the preset feature information of the bank from the database, including the table header name, the start flag of the statement, and the end flag of the statement. The table in the bank statement document is divided into a table header part and a table body part. The table header name is each table header name included in the table header part of the bank, the start flag of the statement is the previous text information of the table body part, and the end flag of the statement is the next text information of the table body part. For example, as Figure 3 shown, the table header names included in the table header part are serial number, summary, currency, cash / remittance, transaction date, transaction amount, account balance, transaction location / memo, counterparty account number and name. The start flag of the statement is the text information corresponding to the last table header name "counterparty account number and name", and the end flag of the statement is the text information corresponding to "generation time". In the original bank statement document, with the upper left corner as the origin of the coordinate system, arrange each text content in the order from top to bottom and in the order from left to right in the same row. Then this order corresponds to the arrangement order of each HTML element in the HTML document. Therefore, in the HTML document, each table header HTML element can be determined according to the table header name, and each HTML element between the HTML elements corresponding to the start flag of the statement and the end flag of the statement is determined as the table body HTML element.

[0114] The data processing method proposed in this embodiment converts the bank statement document into an HTML document, and obtains the HTML elements based on the HTML document. Then, based on the HTML elements and the preset feature information of each preset bank, the bank corresponding to the bank statement document is determined. Thus, based on the preset feature information corresponding to the bank, the table header HTML element and the table body HTML element are determined. This facilitates generating corresponding key-value pairs according to the table header HTML element and the table body HTML element, determining the transaction record information according to the key-value pairs, and finally transforming the bank statement information in the bank statement PDF document into structured transaction data and outputting it as the transaction record information, so as to achieve the purpose of obtaining accurate and unified transaction data according to different bank statement documents, and ensuring the accuracy and effectiveness of the parsing result of the bank statement information.

[0115] In addition, an embodiment of the present invention also proposes a data processing device, which includes: a memory, a processor, and a data processing program stored on the memory and executable on the processor. When the data processing program is executed by the processor, the steps of the data processing method described above are implemented.

[0116] In addition, an embodiment of the present invention also proposes a computer-readable storage medium, on which a data processing program is stored. When the data processing program is executed by a processor, the steps of the data processing method described above are implemented.

[0117] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0118] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0120] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.< / text> < / text> < / text> < / text> < / text> < / text> < / text> < / text> < / text>

Claims

1. A data processing method, characterized in that, The data processing method includes the following steps: Convert the bank statement document into an HTML document, and in the HTML elements of the HTML document, determine the table header HTML elements corresponding to each table header name and the table text HTML elements corresponding to each table text in the bank statement document; Based on the table header HTML elements and the table text HTML elements, determine the position information of each table header name and each table text in the bank statement document; Based on the order of the table text HTML elements in the HTML document, traverse each table text HTML element, and generate a key-value pair corresponding to the currently traversed table text HTML element based on the position information, where the value data of the key-value pair is the current table text corresponding to the currently traversed table text HTML element, and the key data of the key-value pair is the table header name corresponding to the current table text; Determine the statement record information corresponding to the bank statement document based on the key-value pair, and output the statement record information when the traversal of each table text HTML element is completed; Among them, the step of traversing each table text HTML element based on the order of the table text HTML elements in the HTML document and generating a key-value pair corresponding to the currently traversed table text HTML element based on the position information includes: Use the table text corresponding to the currently traversed table text HTML element as the to-be-processed table text, and based on the position information, obtain the table text range corresponding to the to-be-processed table text, where the table text range is the horizontal coordinate range of the to-be-processed table text, and the position information includes the table header start coordinates, table header pixel widths corresponding to each table header name, and the table text start coordinates, table text pixel widths corresponding to each table text; Based on the position information and a first preset range coefficient, obtain the table header range corresponding to each table header name, where the table header range is the horizontal coordinate range corresponding to each table header name; Compare each table header range based on the table text range, and determine whether there is a target table header range that covers the table text range among each table header range; If it exists, generate the key-value pair based on the to-be-processed table text and the table header name corresponding to the target table header range.

2. The data processing method according to claim 1, wherein After the step of comparing each table header range based on the table text range and determining whether there is a target table header range that covers the table text range among each table header range, it further includes: If it does not exist, update the acquisition times of the table header range, and determine a second preset range coefficient based on the acquisition times and a preset ratio; Use the second preset range coefficient as the first preset range coefficient, and return to execute the step of obtaining the table header range corresponding to each table header name based on the position information and the first preset range coefficient.

3. The data processing method according to claim 1, wherein The step of using the table text corresponding to the currently traversed table text HTML element as the to-be-processed table text and obtaining the table text range corresponding to the to-be-processed table text based on the position information includes: Based on the table text HTML element, obtain the starting coordinate of the table text corresponding to the to-be-processed table text and the pixel width of the table text; Based on the starting coordinate of the table text and the pixel width of the table text, determine the table text range.

4. The data processing method according to claim 1, wherein The step of obtaining the header range corresponding to each header name based on the position information and the first preset range coefficient includes: Based on the header HTML element, obtain the starting coordinate of each header name and the pixel width of the header; Based on the starting coordinate of the header and the pixel width of the header, determine the central position information corresponding to each header name; Based on the central position information and the first preset range coefficient, determine the header range.

5. The data processing method according to claim 1, characterized in that The step of determining the transaction record information corresponding to the bank statement document based on the key-value pair and outputting the transaction record information when traversing is completed for each of the table text HTML elements includes: If the key-value pair is not the first generated key-value pair, then use the key-value pair as the first key-value pair, use the previous key-value pair adjacent to the first key-value pair as the second key-value pair, and match the first header name corresponding to the first key-value pair with the second header name corresponding to the second key-value pair; If the serial number corresponding to the first header name is 1 and the first header name is different from the second header name, then create a new first transaction record and store the key-value pair in the first transaction record, and update the transaction record information based on the first transaction record, or; If the serial number corresponding to the first header name is not 1 and the first header name is different from the second header name, then store the key-value pair in the second transaction record containing the second key-value pair, and update the transaction record information based on the second transaction record, or; If the first header name is the same as the second header name, then update the second table text corresponding to the second key-value pair based on the first table text corresponding to the first key-value pair, and update the transaction record information based on the updated second table text; When traversing is completed for each of the table text HTML elements, output the transaction record information.

6. The data processing method according to claim 5, characterized in that, Before the step of, if the key-value pair is not the first generated key-value pair, then use the key-value pair as the first key-value pair, use the previous key-value pair adjacent to the first key-value pair as the second key-value pair, and match the first header name corresponding to the first key-value pair with the second header name corresponding to the second key-value pair, further includes: If the key-value pair is the first generated key-value pair, then create a new first transaction record and store the key-value pair in the first transaction record; Update the transaction record information based on the first transaction record.

7. The data processing method according to any one of claims 1 to 6, characterized in that The step of converting the bank statement document into an HTML document and determining, in the HTML elements of the HTML document, the header HTML elements corresponding to each header name in the bank statement document and the table text HTML elements corresponding to each table text includes: Convert the bank statement document into an HTML document, and obtain the HTML elements based on the HTML document; Determine the bank corresponding to the bank statement document based on the HTML element and the preset feature information of each preset bank; Determine the table header HTML element and the table text HTML element based on the preset feature information corresponding to the bank.

8. A data processing device, characterized in that, The data processing device includes: a memory, a processor, and a data processing program stored on the memory and executable on the processor. When the data processing program is executed by the processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium, characterized in that, A data processing program is stored on the readable storage medium. When the data processing program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.

Images