A financial bill analysis method and system based on image recognition technology

Abstract

The present invention provides a financial bill analysis method and system based on image recognition technology, which relates to the field of data processing technology. The method includes: obtaining a financial bill image; constructing a text box dynamic detection model; inputting the financial bill image into the text box dynamic detection model, and outputting a target text area box; performing text extraction on the target text area box to obtain a group of text contents corresponding to each text area box; performing non-modification content extraction on each group of text contents in turn according to the parsing target to obtain the parsing content of the parsing target, wherein the parsing target includes a transaction object, a transaction address, a transaction category, a transaction date, and a transaction amount; outputting the parsing target and the corresponding parsing content in pairs in turn to obtain the analysis result of the financial bill. The content extraction accuracy of the financial bill and the consistency of the analysis result with the original financial bill are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a financial bill analysis method and system based on image recognition technology. Background Art

[0002] Financial bill analysis refers to the process of automatically extracting, classifying, checking and recording data of various financial bills (such as invoices, receipts, expense reports, etc.) through technical means. This analysis process usually involves image recognition, OCR (optical character recognition), natural language processing and other technologies to extract key information on the bill (such as date, amount, transaction object, etc.) and convert it into structured data for further processing and analysis. Through financial bill analysis, enterprises can manage and process a large number of financial bills more efficiently, reduce errors in manual input, and improve the speed and accuracy of data processing, thereby optimizing financial management processes.

[0003] The necessity of financial bill analysis lies in that it significantly improves the efficiency and accuracy of corporate financial management. Traditional manual processing is time-consuming and error-prone, especially when dealing with a large number of bills. The manual input and verification process is cumbersome and error-prone. Automated bill analysis technology can quickly and accurately extract and record financial data, reduce human intervention and errors, and ensure the integrity and consistency of financial data. In addition, automated analysis helps companies to timely discover and prevent financial risks, improve financial transparency and compliance, and thus enhance the company's decision-making support capabilities and overall competitiveness.

[0004] However, in the existing financial bill analysis process, the various factors of the financial bill form are not taken into consideration and the content is directly extracted. This is not only inefficient but also easily affected by complex and fuzzy backgrounds. The extraction results often contain many errors, which are very serious for rigorous financial data. Summary of the invention

[0005] In order to solve the problem in the prior art that in the existing financial bill analysis process, various factors of the financial bill form are not taken into consideration, and the content is directly extracted, which is not only inefficient but also easily affected by complex and fuzzy backgrounds, and the extraction results often have many errors. Such errors are very serious technical problems for rigorous financial data. The present invention provides a financial bill analysis method and system based on image recognition technology.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect

[0008] An embodiment of the present invention provides a financial bill analysis method based on image recognition technology, comprising:

[0009] S1: Acquire financial bill image;

[0010] S2: construct a text box dynamic detection model, wherein the text box dynamic detection model includes a multi-feature extraction module, a feature enhancement module, a text proposal generation module and a text detection refinement module, the multi-feature extraction module is connected to the text detection refinement module, the multi-feature extraction module is connected to the text detection refinement module through the feature enhancement module and the text proposal generation module in turn, the multi-feature extraction module is connected to the image feature map of the financial bill image, wherein the feature enhancement module includes a first convolutional layer, a second convolutional layer and a third deconvolutional layer connected to the image feature map of the financial bill image through a residual network, the outputs of each convolutional layer are output in series, and the text detection refinement module includes an adaptive weight RoI pooling layer, wherein the adaptive weight RoI pooling layer has multiple pooling sizes, each pooling size has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module;

[0011] S3: Input the financial bill image into the text box dynamic detection model and output the target text area box;

[0012] S4: extracting text from the target text area frame to obtain a set of text contents corresponding to each text area frame;

[0013] S5: performing non-modification content extraction on each group of text content in turn according to the parsing target to obtain the parsed content of the parsing target, wherein the parsing target includes the transaction object, transaction address, transaction category, transaction date and transaction amount;

[0014] S6: Output the analysis target and the corresponding analysis content in pairs in sequence to obtain the analysis result of the financial bill.

[0015] Second aspect

[0016] An embodiment of the present invention provides a financial bill analysis system based on image recognition technology, comprising:

[0017] processor;

[0018] A memory having computer-readable instructions stored therein, which, when executed by a processor, implements the financial bill analysis method based on image recognition technology as in the first aspect.

[0019] The third aspect

[0020] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the financial document analysis method based on image recognition technology as described in the first aspect is implemented.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0022] In the present invention, by constructing a dynamic text box detection model, the feature map of the financial bill image is pooled through a variety of pooling sizes, and each pooling size has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module, so that the pooling operation can flexibly adapt to texts of different sizes and shapes, and adaptively adjust the target text area box, while reducing the output processing amount of subsequent text extraction, effectively reducing the influence of complex background, thereby improving the accuracy and efficiency in text detection, especially when processing very small or irregularly shaped text, it can more effectively capture the necessary text feature information, avoiding the fixed pooling size in the traditional method. In addition, a multi-layer convolution layer including a deconvolution layer is introduced in the feature enhancement module. The output of each convolution layer is connected in series, which helps to distinguish the foreground object from the background of the original image feature map, and provides a data basis for the subsequent text region proposal, thereby improving the accuracy of subsequent text extraction. In addition, in the process of content extraction, content extraction is performed in a non-modified manner, and the original data of the financial bill is strictly retained to ensure the rigor and unmodifiable nature of the financial bill analysis. This automated financial bill analysis method can protect the security of financial bills while ensuring compliance with financial data processing specifications, and eliminate the uncontrollable factors of subjective modification that may occur in the manual sorting of financial bills. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flowchart of a financial bill analysis method based on image recognition technology provided by an embodiment of the present invention;

[0025] Figure 2 A structural schematic diagram of a text box dynamic detection model provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of the structure of a financial document analysis system based on image recognition technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0028] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0029] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0030] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] Reference Manual Attached Figure 1 , showing a flow chart of a financial bill analysis method based on image recognition technology provided by an embodiment of the present invention.

[0033] The embodiment of the present invention provides a financial bill analysis method based on image recognition technology, which can be implemented by a financial bill analysis device based on image recognition technology, and the financial bill analysis device based on image recognition technology can be a terminal or a server. The processing flow of the financial bill analysis method based on image recognition technology may include the following steps:

[0034] S1: Acquire a financial document image.

[0035] Among them, financial documents include electronic versions collected from paper versions and original electronic versions.

[0036] In a possible implementation manner, after S1, the method further includes:

[0037] The financial document image is preprocessed including image tilt correction.

[0038] It should be noted that the electronic version obtained by collecting the paper version may have different defects due to different collection methods, such as wrinkles, tilt or shadow, which greatly increases the difficulty of extracting the content of the financial document. Tilt correction and image enhancement can greatly improve the accuracy and efficiency of subsequent financial document extraction.

[0039] In a possible implementation, the financial document image is preprocessed including image tilt correction, specifically including:

[0040] Perform edge detection on the financial document image to obtain multiple edge points.

[0041] Use Hough transform to detect edge points and obtain the straight line (ρ, θ) in the financial bill image:

[0042] ρ=x cos(θ)+y sin(θ)

[0043] Among them, ρ represents the distance from the origin to the straight line, θ represents the rotation angle of the straight line relative to the X-axis, where the origin is the upper left corner of the financial document image, (x, y) represents the pixel coordinates of the financial document image, and sin and cos represent the calculated sine and cosine values ​​respectively.

[0044] Perform image tilt correction on financial document images according to the rotation angle:

[0045]

[0046] α=-θ

[0047] Wherein, (x', y') represents the pixel coordinates of the corrected financial document image.

[0048] It should be noted that the image preprocessing step significantly improves the precision and accuracy of subsequent text detection. The combined use of edge detection and Hough transform enables the straight line features in the image to be accurately identified, thereby effectively correcting the tilt of the image. Edge detection extracts edge points in the image, detects the straight line formed by these edge points through Hough transform, calculates the parameters ρ and θ of the straight line, and then rotates the image according to these parameters so that the text part in the image is arranged horizontally. In this way, not only the image tilt problem caused by the shooting angle and position is eliminated, but also the impact of defects such as wrinkles and shadows on text recognition is reduced. Through this step, the text area that may have been difficult to identify due to tilt and deformation becomes clearer and more regular, greatly reducing the errors in the subsequent text extraction process, and ensuring the accuracy and reliability of the extraction of financial bill content. On the whole, this preprocessing method enhances the robustness and efficiency of the entire text detection system, enabling it to maintain high performance in complex practical application scenarios, and can further enhance the precision and accuracy of text detection.

[0049] Refer to the instruction manual Figure 2 , showing a structural schematic diagram of a text box dynamic detection model provided by an embodiment of the present invention.

[0050] Figure 2 In it, conv represents convolution operation, Res represents residual network, and ROI represents region of interest. ROI refers to a specific area selected in an image. Here, ROI refers to multiple proposed text proposal boxes.

[0051] S2: Build a text box dynamic detection model.

[0052] Among them, the text box dynamic detection model includes a multi-feature extraction module, a feature enhancement module, a text proposal generation module and a text detection refinement module. The multi-feature extraction module is connected to the text detection refinement module, and the multi-feature extraction module is connected to the text detection refinement module through the feature enhancement module and the text proposal generation module in turn. The multi-feature extraction module is connected to the image feature map of the financial document image.

[0053] Among them, the feature enhancement module includes a first convolution layer, a second convolution layer and a third deconvolution layer connected to the image feature map of the financial document image through a residual network, and the outputs of each convolution layer are output in series. The text detection refinement module includes an adaptive weight RoI pooling layer, wherein the adaptive weight RoI pooling layer has multiple pooling sizes, and each pooling size has a pooling size weight that is adaptive to the size of the text proposal box generated by the text proposal generation module.

[0054] Specifically, through the design of the multi-feature extraction module, the multi-dimensional feature information in the financial document image can be effectively captured. The feature enhancement module uses the residual network and multi-layer convolutional layers to extract and enhance the image features layer by layer, thereby improving the model's recognition ability of text areas under complex backgrounds. Secondly, the text proposal generation module can generate accurate text proposal boxes to ensure the initial accuracy of text detection. Most importantly, the introduction of the adaptive weight RoI pooling layer enables the model to dynamically adjust the pooling size according to the size of the text proposal box, thereby maintaining high detection accuracy and efficiency when processing texts of different sizes and shapes. This adaptive mechanism can maximize the capture of necessary text feature information, especially when detecting small or irregularly shaped texts, and avoid the problem of information loss that may be caused by fixed pooling size.

[0055] In one possible implementation, the multi-feature extraction module includes a plurality of parallel residual network units with different depths, wherein the residual network unit includes a residual block of a preset depth, a bottleneck convolution layer, and a fixed channel output layer connected in sequence.

[0056] It should be noted that those skilled in the art can set the preset depth according to actual needs, and the present invention is not limited thereto.

[0057] Specifically, the multi-feature extraction module has residual networks of different depths. The residual networks in the multi-feature extraction module include Res11, Res16, Res21, Res27 and Res33, which means that the features extracted by the residual blocks of different depths of the residual network are used as output, that is, the outputs of the 11th, 16th, 21st, 27th and 33rd layers of the residual network are used as the extracted features of the image features, and then the output is convolved with 1x1 to output a feature map with 256 channels, that is, for each residual network, the connection mode of Resxx-conv-1x1-256 is used to output the extracted features, and the features output from the xxth layer of ResNet are processed by a convolution layer using a 1x1 convolution kernel to output a feature map of 256 channels, and each residual network is extracted in parallel. By using the convolution layer as the bottleneck convolution, the intermediate feature maps are compressed along the channel dimension. The reused intermediate feature maps are consistent in spatial size and can directly provide deep supervision information for some intermediate residual blocks. This parallelizes residual learning instead of traditional serial residual learning, which can effectively improve computational efficiency.

[0058] In a possible implementation, the first convolution layer is a 1x3 convolution layer, the second convolution layer is a 3x3 convolution layer, and the third deconvolution layer includes a pooling layer, a 1x1 convolution layer, and a deconvolution layer in sequence.

[0059] It should be noted that in the field of general object detection, only 3×3 convolution sliding window features are always used on some intermediate layers to generate region proposals, which is insufficient for text region proposals. Since the width of most words or text lines is usually much larger than the height, in other words, their aspect ratio is usually much larger than 1. At the same time, high-level semantic features have a larger receptive field and contain more contextual information. The residual network depth can be selected as Res30, and based on the output of the residual block, the first convolution layer is designed to be a 1x3 convolution layer, the second convolution layer is a 3x3 convolution layer, and the third deconvolution layer includes a pooling layer, a 1x1 convolution layer, and a deconvolution layer, which are connected in series after feature extraction. This helps to distinguish foreground objects from the background of the original image feature map, provide a data basis for subsequent text region proposals, and improve the accuracy of text proposals.

[0060] In actual application, the Res30 layer of the network stem is selected as the input layer of the feature enhancement module. Two branches are added on the basis of the traditional 3x3 convolution layer. One is a task-specific 1×3 convolution layer with text characteristics, and the other contains maximum pooling, 1×1 convolution, and deconvolution layers. The application of the deconvolution layer can completely eliminate the checkerboard artifacts of the traditional deconvolution operation, and it can make the text features more prominent and extract more contextual information. Then, we concatenate the original 3×3 convolution sliding window features with the output of the deconvolution layer and the 1×3 convolution layer.

[0061] S3: Input the financial document image into the text box dynamic detection model and output the target text area box.

[0062] In actual application, whether financial documents are electronically printed or handwritten, the fonts may fade and become blurred due to the length of time they are placed or other reasons before they are photographed and uploaded, which greatly reduces the accuracy of text content extraction. The extracted target text area box can greatly reduce the amount of data processing in the subsequent text extraction process, and accordingly, it also reduces the impact of complex and blurred backgrounds on the accuracy of text extraction, complies with the rigorous data collection specifications for financial data, shields a large amount of irrelevant interference, narrows the search range for subsequent text content extraction, and greatly improves the efficiency and accuracy of text extraction.

[0063] In a possible implementation, S3 specifically includes:

[0064] S301: Select multiple anchor points through an anchor point mechanism.

[0065] Among them, anchors are predefined reference boxes that cover different areas at different locations in the image. These anchors are candidate object detection regions and are used to train the detection model to recognize and locate objects in the image. At each point on the feature map, multiple anchors are placed with different scales and aspect ratios. Scale refers to the size of the anchor, usually in pixels. In the context of text detection, the scale reflects the size of the area covered by the anchor box. Different scales can help the model detect text of different sizes. For example, a smaller scale (such as 32 pixels) is suitable for detecting small text, while a larger scale (such as 416 pixels) is suitable for detecting entire text lines or larger text blocks. The aspect ratio is the ratio of the width to the height of the anchor. Since text is usually in the form of strips, its aspect ratio is usually greater than 1 (that is, the width is greater than the height). By setting multiple aspect ratios, text regions of different shapes can be more accurately adapted and detected. For example, an anchor with an aspect ratio of 6 is suitable for detecting very long and thin text lines.

[0066] Specifically, it is almost impossible for a text word to occupy the entire input image, so I chose five scales (32, 64, 112, 192 and 304) and four aspect ratios (1, 2, 3 and 4) for the anchor points here.

[0067] S302: applying the anchor point to the high-level feature map extracted by the feature enhancement module, and generating a text proposal box through the text proposal generation module according to the high-level feature map after applying the anchor point:

[0068] x=x 0 +w 0 ×Δx

[0069] y=y 0 +h 0 ×Δy

[0070] w=w 0 ×e Δw

[0071] h=h 0 ×e Δh

[0072] Among them, (x 0 ,y 0 ,w 0 ,h 0 ) represents the center coordinates, width and height of the anchor point, (x, y, w, h) represents the center coordinates, width and height of the text proposal box, (Δx, Δy, Δw, Δh) represents the center coordinate offset, width offset and height offset of the text proposal box relative to the corresponding anchor point, and e represents the base of the natural logarithm.

[0073] In a possible implementation manner, after S302, the method further includes:

[0074] Positive sample mining is performed on each text proposal box with an intersection-over-union ratio of 0.7.

[0075] The text box dynamic detection model is trained through the mined positive samples.

[0076] It should be noted that adding positive sample mining with an intersection-over-union ratio of 0.7 and training the model can help solve the sample imbalance problem. This method can ensure that the selected text proposal box is closer to the actual target text area and improve the quality of the training data. By training the model with mined positive samples, the model's recognition ability for diverse text areas can be enhanced, and the model's generalization performance and detection accuracy can be improved. Ultimately, this method can significantly improve the stability and accuracy of the text box dynamic detection model in various complex backgrounds, effectively solve the sample imbalance problem, and improve the training effect of the dynamic monitoring model.

[0077] S303: Divide each text proposal box into multiple grids according to different pooling sizes.

[0078] Specifically, the pooling sizes can be selected as 3x3, 7x7, 3x8, and 3x11, which cover various arrangements of text.

[0079] S304: Generate a grid score map at each pooling size using the high-level feature map.

[0080] S305: Calculate the influence weight of each pooling size according to the grid score map:

[0081]

[0082] Among them, S a (Θ) represents the pooling score corresponding to the grid score map of the ath pooling size, e represents the base of the natural logarithm, r a (i,j|Θ) represents the pooling value of the grid (i,j) in the grid score map obtained by the ath pooling size, r a (Θ) represents the pooling score of the text proposal box under the a-th pooling size, p and q represent the horizontal and vertical parameters of the pooling size respectively, A represents the type of pooling size, Θ represents the network parameter set of the text box dynamic detection model, n represents the number of pixels in the cell bin where the grid (i, j) is located, (x', y') represents the cell coordinates, (x 0 ',y 0 ') represents the coordinates of the upper left corner of the text proposal box in the high-level feature map, W a (Θ) represents the influence weight of the ath pooling size, that is, the pooling size weight.

[0083] S306: Accumulate the product of the pooling scores at each pooling size and the corresponding influence weight to obtain the text proposal box pooling score:

[0084]

[0085] Among them, S represents the pooling score of the text proposal box.

[0086] S307: Select the text proposal box with the highest text proposal box pooling score as the target text area box output.

[0087] Specifically, the accuracy and flexibility of text detection are effectively improved through the anchor mechanism and multi-scale pooling method. First, by selecting anchors of multiple scales and aspect ratios, it can better adapt to text areas of different sizes and shapes, thereby covering a wider range of text detection needs. Secondly, the text proposal generation module combines these anchors with high-level feature maps to generate more accurate text proposal boxes. The use of multiple pooling sizes enables the model to effectively extract and score features for texts of different layouts, and further optimizes the detection accuracy through adaptive pooling size weights. The calculation of the pooling score map and the introduction of the pooling size weight ensure that each text proposal box can be flexibly adjusted according to the actual situation, and improve the adaptability to complex backgrounds and diverse text forms. Finally, the text proposal box with the highest pooling score is selected to ensure the optimal output of the target text area box. This method not only performs well in processing small fonts and irregularly shaped text, but also reduces the impact of complex backgrounds on detection accuracy, thereby achieving efficient and accurate text detection in various practical application scenarios.

[0088] It should be noted that text regions are usually smaller than general objects, and the aspect ratio of text words is exaggerated. For example, if there are two text samples with spatial sizes of 48×48 and 48×256 in the input image, after the forward propagation process, the corresponding text regions will be 3×3 and 3×16 in the feature map Res30. The prior art usually unreasonably applies a 7×7 position-sensitive RoI pooling operation. The adaptive weight RoI pooling layer has multiple pooling sizes, each of which has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module, that is, different pooling sizes are suitable for different text regions with different spatial sizes and aspect ratios. The most suitable pooling size will have the highest adaptive weight, and it will naturally contribute the most to the final text score. The higher the pooling score of the text proposal box, the more the anchor position is responsively corrected to the proposal box. The target text area box obtained in the end is tightly surrounded by the text area, which can exclude a large amount of irrelevant content, reduce the data processing amount of extracting text content, and increase the accuracy of subsequent text content extraction.

[0089] S4: extracting text from the target text area frame to obtain a group of text contents corresponding to each text area frame.

[0090] Specifically, the text content in the target text area may be extracted by an OCR algorithm.

[0091] S5: performing non-modification content extraction on each group of text contents in turn according to the parsing target to obtain the parsed content of the parsing target.

[0092] Among them, the parsing targets include transaction objects, transaction addresses, transaction categories, transaction dates and transaction amounts.

[0093] It should be noted that during the financial document analysis process, the original text of the financial document is retained to eliminate the generative content deviation caused by automated extraction, and the financial content is concisely displayed in the original form. The analysis results are accurate and highly reliable.

[0094] In a possible implementation, S5 specifically includes:

[0095] S501: Obtain synonymous resolution targets of each resolution target, wherein the resolution target and the synonymous resolution targets form a resolution target set.

[0096] Specifically, synonym traversal based on natural language processing technology is directly performed in the financial database. During the traversal, synonyms of the parsing target are obtained. The financial database may specifically be Morningstar database and Bloomberg Financial Services database, etc.

[0097] S502: Execute the keyword trigger strategy using the parsing target set to identify the parsing targets of the text content.

[0098] S503: Calculate the cosine similarity between the identified parsing target and the adjacent text content of the parsing target, wherein the adjacent text content is the text content of the preset bytes after the parsing target:

[0099]

[0100] Among them, cos(v target ,v context ) represents the parsing target vector v target and the adjacent text content vector v context The cosine similarity of , || || indicates the calculation of the Euclidean norm of the vector.

[0101] Specifically, by calculating the cosine similarity between the parsed target and the adjacent text content, it helps to accurately identify and match related content. Cosine similarity can effectively evaluate the similarity of two texts by measuring the angle between vectors, regardless of the length of the text. This is particularly important in processing financial documents, because the contextual information of adjacent texts is crucial to the accurate extraction of the parsed target. Through this method, the accuracy and robustness of target recognition can be improved, erroneous parsing can be reduced, and the high accuracy and consistency of the extraction results can be ensured.

[0102] It should be noted that those skilled in the art can set the size of the preset bytes according to actual needs, and the present invention does not limit this.

[0103] S504: retaining adjacent text contents with a cosine similarity greater than a preset value as parsed contents of the parsing target.

[0104] It should be noted that those skilled in the art can set the size of the preset cosine similarity according to actual needs, and the present invention is not limited here.

[0105] In actual application, natural language processing technology is used to perform synonym traversal in the financial database and build a parsing target set to ensure that all possible synonymous expressions are covered, thereby improving the comprehensiveness and accuracy of the parsing. The parsing target is identified by the keyword trigger strategy for the text content, and the cosine similarity calculation is combined to accurately match the parsing target with its adjacent text content to ensure that the extracted content is highly relevant to the parsing target. This method not only improves the accuracy of parsing target identification, but also effectively filters out irrelevant information, ultimately ensuring the accuracy and reliability of the parsing results. This intelligent processing method reduces errors caused by human intervention and improves the automation level and efficiency of financial data parsing.

[0106] S6: Output the analysis target and the corresponding analysis content in pairs in sequence to obtain the analysis result of the financial bill.

[0107] In a possible implementation manner, after S6, the method further includes:

[0108] The analysis results are stored in encrypted form.

[0109] It is understandable that encrypting and storing the analysis results can significantly improve the security and confidentiality of the data and prevent unauthorized access and data leakage. Financial data usually involves sensitive information. Encrypted storage can ensure the integrity and confidentiality of the data during storage and transmission, in line with industry standards. At the same time, encrypted storage also provides higher security for the subsequent management and auditing of the data, enhancing the overall security and reliability of the system.

[0110] In actual application, there are various types of financial documents. Manual processing is inefficient and easily interfered by subjective factors, which may lead to man-made financial losses. Through the financial document analysis method described above, the financial data is automatically analyzed, the efficiency is improved and the accuracy is higher. It can not only assist financial personnel in efficient financial processing, but also in the automation process, the parsed content of the financial document is irreversible, which can effectively avoid man-made financial losses in the financial analysis process, reduce financial management costs and risks, and effectively improve the accuracy of content extraction of financial documents and the consistency of analysis results with the original financial documents.

[0111] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0112] In the present invention, by constructing a dynamic text box detection model, the feature map of the financial bill image is pooled through a variety of pooling sizes, and each pooling size has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module, so that the pooling operation can flexibly adapt to texts of different sizes and shapes, and adaptively adjust the target text area box, while reducing the output processing amount of subsequent text extraction, effectively reducing the influence of complex background, thereby improving the accuracy and efficiency in text detection, especially when processing very small or irregularly shaped text, it can more effectively capture the necessary text feature information, avoiding the fixed pooling size in the traditional method. In addition, a multi-layer convolution layer including a deconvolution layer is introduced in the feature enhancement module. The output of each convolution layer is connected in series, which helps to distinguish the foreground object from the background of the original image feature map, and provides a data basis for the subsequent text region proposal, thereby improving the accuracy of subsequent text extraction. In addition, in the process of content extraction, content extraction is performed in a non-modified manner, and the original data of the financial bill is strictly retained to ensure the rigor and unmodifiable nature of the financial bill analysis. This automated financial bill analysis method can protect the security of financial bills while ensuring compliance with financial data processing specifications, and eliminate the uncontrollable factors of subjective modification that may occur in the manual sorting of financial bills.

[0113] Reference Manual Attached Figure 3 , showing a structural schematic diagram of a financial bill analysis system based on image recognition technology provided by the present invention.

[0114] The present invention further provides a financial bill analysis system 20 based on image recognition technology, which is applied to the above-mentioned financial bill analysis method based on image recognition technology, and comprises:

[0115] Processor 201.

[0116] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , the financial bill analysis method based on image recognition technology in the method embodiment is implemented.

[0117] The financial document analysis system 20 based on image recognition technology provided by the present invention can execute the above-mentioned financial document analysis method based on image recognition technology and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0118] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0119] In the present invention, by constructing a dynamic text box detection model, the feature map of the financial bill image is pooled through a variety of pooling sizes, and each pooling size has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module, so that the pooling operation can flexibly adapt to texts of different sizes and shapes, and adaptively adjust the target text area box, while reducing the output processing amount of subsequent text extraction, effectively reducing the influence of complex background, thereby improving the accuracy and efficiency in text detection, especially when processing very small or irregularly shaped text, it can more effectively capture the necessary text feature information, avoiding the fixed pooling size in the traditional method. In addition, a multi-layer convolution layer including a deconvolution layer is introduced in the feature enhancement module. The output of each convolution layer is connected in series, which helps to distinguish the foreground object from the background of the original image feature map, and provides a data basis for the subsequent text region proposal, thereby improving the accuracy of subsequent text extraction. In addition, in the process of content extraction, content extraction is performed in a non-modified manner, and the original data of the financial bill is strictly retained to ensure the rigor and unmodifiable nature of the financial bill analysis. This automated financial bill analysis method can protect the security of financial bills while ensuring compliance with financial data processing specifications, and eliminate the uncontrollable factors of subjective modification that may occur in the manual sorting of financial bills.

[0120] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0121] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0122] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0123] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0124] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0125] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0126] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0131] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a financial bill analysis method based on image recognition technology as described in the method embodiment.

[0133] A computer-readable storage medium provided by the present invention can implement the steps and effects of the financial document analysis method based on image recognition technology in the above method embodiment. To avoid repetition, the present invention will not go into details.

[0134] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0135] In the present invention, by constructing a dynamic text box detection model, the feature map of the financial bill image is pooled through a variety of pooling sizes, and each pooling size has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module, so that the pooling operation can flexibly adapt to texts of different sizes and shapes, and adaptively adjust the target text area box, while reducing the output processing amount of subsequent text extraction, effectively reducing the influence of complex background, thereby improving the accuracy and efficiency in text detection, especially when processing very small or irregularly shaped text, it can more effectively capture the necessary text feature information, avoiding the fixed pooling size in the traditional method. In addition, a multi-layer convolution layer including a deconvolution layer is introduced in the feature enhancement module. The output of each convolution layer is connected in series, which helps to distinguish the foreground object from the background of the original image feature map, and provides a data basis for the subsequent text region proposal, thereby improving the accuracy of subsequent text extraction. In addition, in the process of content extraction, content extraction is performed in a non-modified manner, and the original data of the financial bill is strictly retained to ensure the rigor and unmodifiable nature of the financial bill analysis. This automated financial bill analysis method can protect the security of financial bills while ensuring compliance with financial data processing specifications, and eliminate the uncontrollable factors of subjective modification that may occur in the manual sorting of financial bills.

[0136] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0137] There are a few points to note:

[0138] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0139] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0140] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0141] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A financial bill analysis method based on image recognition technology, characterized in that: include: S1: Acquire financial bill image; S2: Construct a text box dynamic detection model, wherein the text box dynamic detection model includes a multi-feature extraction module, a feature enhancement module, a text proposal generation module and a text detection refinement module, the multi-feature extraction module is connected to the text detection refinement module, the multi-feature extraction module is connected to the text detection refinement module through the feature enhancement module and the text proposal generation module in turn, the multi-feature extraction module is connected to the image feature map of the financial bill image, wherein the feature enhancement module includes a first convolution layer, a second convolution layer and a third deconvolution layer connected to the image feature map of the financial bill image through a residual network, the outputs of each convolution layer are output in series, and the text detection refinement module includes an adaptive weight RoI pooling layer, wherein the adaptive weight RoI pooling layer has multiple pooling sizes, each pooling size has a pooling size weight that is adaptive to the text proposal box size generated by the text proposal generation module; S3: inputting the financial bill image into the text frame dynamic detection model, and outputting a target text area frame; S4: extracting text from the target text area frame to obtain a group of text contents corresponding to each text area frame; S5: performing non-modification content extraction on each group of text content in turn according to the parsing target to obtain the parsed content of the parsing target, wherein the parsing target includes the transaction object, transaction address, transaction category, transaction date and transaction amount; S6: Outputting the analysis target and the corresponding analysis content in pairs in sequence to obtain the analysis result of the financial bill; Wherein, the S3 specifically includes: S301: Select multiple anchor points through an anchor point mechanism; S302: applying the anchor point to the high-level feature map extracted by the feature enhancement module, and generating a text proposal box through the text proposal generation module according to the high-level feature map after the anchor point is applied: x=x0+w0×Δx y=y0+h0×Δy w=w0×e Δw h=h0×e Δh Wherein, (x0, y0, w0, h0) represents the center coordinates, width and height of the anchor point, (x, y, w, h) represents the center coordinates, width and height of the text proposal box, (Δx, Δy, Δw, Δh) represents the center coordinate offset, width offset and height offset of the text proposal box relative to the corresponding anchor point, and e represents the base of the natural logarithm; S303: Divide each text proposal box into multiple grids according to different pooling sizes; S304: Generate a grid score map at each pooling size using the high-level feature map; S305: Calculate the influence weight of each pooling size according to the grid score map: Among them, S a (Θ) represents the pooling score corresponding to the grid score map of the ath pooling size, e represents the base of the natural logarithm, r a (i,j|Θ) represents the pooling value of the grid (i,j) in the grid score map obtained by the ath pooling size, r a (Θ) represents the pooling score of the text proposal box under the a-th pooling size, p and q represent the horizontal and vertical parameters of the pooling size respectively, A represents the type of pooling size, Θ represents the network parameter set of the text box dynamic detection model, n represents the number of pixels in the cell bin where the grid (i, j) is located, (x', y') represents the cell coordinates, (x0', y0') represents the coordinates of the upper left corner of the text proposal box in the high-level feature map, W a (Θ) represents the influence weight of the a-th pooling size, i.e., the pooling size weight; S306: Accumulate the product of the pooling scores at each pooling size and the corresponding influence weight to obtain the text proposal box pooling score: Among them, S represents the pooling score of the text proposal box; S307: Select the text proposal box with the highest pooling score of the text proposal boxes as the target text area box for output.

2. The financial bill analysis method based on image recognition technology according to claim 1 is characterized in that: After S1, the following is also included: The financial document image is preprocessed including image tilt correction.

3. The financial bill analysis method based on image recognition technology according to claim 2 is characterized in that: The preprocessing of the financial document image including image tilt correction specifically includes: Performing edge detection on the financial bill image to obtain a plurality of edge points; The edge points are detected using Hough transform to obtain the straight line (ρ, θ) in the financial bill image: ρ=xcos(θ)+ysin(θ) Wherein, ρ represents the distance from the origin to the straight line, θ represents the rotation angle of the straight line relative to the X-axis, wherein the origin is the upper left corner of the financial document image, (x, y) represents the pixel coordinates of the financial document image, and sin and cos represent the calculated sine value and cosine value respectively; Perform image tilt correction on the financial bill image according to the rotation angle: α=-θ Wherein, (x', y') represents the pixel coordinates of the corrected financial document image.

4. The financial bill analysis method based on image recognition technology according to claim 1 is characterized in that: The multi-feature extraction module includes a plurality of parallel residual network units with different depths, wherein the residual network unit includes a residual block of a preset depth, a bottleneck convolution layer and a fixed channel output layer connected in sequence.

5. The financial bill analysis method based on image recognition technology according to claim 1 is characterized in that: The first convolution layer is a 1x3 convolution layer, the second convolution layer is a 3x3 convolution layer, and the third deconvolution layer includes a pooling layer, a 1x1 convolution layer, and a deconvolution layer in sequence.

6. The financial bill analysis method based on image recognition technology according to claim 1 is characterized in that: After S302, the method further includes: Perform positive sample mining on each text proposal box with an intersection-over-union ratio of 0.7; The text box dynamic detection model is trained by mining positive samples.

7. The financial bill analysis method based on image recognition technology according to claim 1 is characterized in that: The S5 specifically includes: S501: Acquire synonymous resolution targets of each resolution target, wherein the resolution targets and the synonymous resolution targets form a resolution target set; S502: Executing a keyword triggering strategy using the parsing target set to identify parsing targets for the text content; S503: Calculate the cosine similarity between the identified parsing target and the adjacent text content of the parsing target, wherein the adjacent text content is the text content of the preset bytes after the parsing target: Among them, cos(v target ,v context ) represents the parsing target vector v target and the adjacent text content vector v context The cosine similarity of || || indicates the Euclidean norm of the calculated vector; S504: retaining adjacent text contents with a cosine similarity greater than a preset value as parsed contents of the parsing target.

8. The financial bill analysis method based on image recognition technology according to claim 1 is characterized in that: After S6, the method further includes: The analysis result is encrypted and stored.

9. A financial bill analysis system based on image recognition technology, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the financial document analysis method based on image recognition technology as described in any one of claims 1 to 8 is implemented.

Images