Financial form identification method, system, device and medium

Abstract

The invention relates to a financial form identification method, system and device and a medium. The method comprises the following steps: acquiring a financial form to be identified; according to a text detection algorithm based on segmentation, feature maps of different scales are extracted from the financial form and fused, and a text area in the financial form is recognized and extracted from the fused feature maps; extracting text sequence features in the text region, performing transcription processing on the extracted text sequence features based on an attention mechanism, and extracting a text sequence in the text region; encoding and semantic enhancement are carried out on the text sequence to obtain a comprehensive vector sequence, the comprehensive vector sequence is decoded, and the named entities in the financial form are recognized. According to the invention, automatic and refined extraction and structured processing of financial form information are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of financial information extraction, and in particular to a method, system, device and medium for identifying a financial form. Background Art

[0002] With the rapid development of the financial industry and technological progress, the demand for digital financial services continues to grow. In the face of an increasingly information-based financial system, it is crucial to improve overall operational efficiency and risk management and ensure the safety of user assets. Therefore, when processing financial data, it is particularly important to record and analyze user trading behaviors, changes in the market environment, and risk warning information. The collection and processing of this information relies on image recognition technology, combined with deep learning algorithms and computer vision technology, which can accurately identify and analyze information in different financial scenarios, thereby providing real-time financial market data around the clock and playing a key role in risk control and early warning.

[0003] However, current form recognition technologies in the financial field generally use traditional template matching solutions, rule-based text extraction solutions, or hybrid feature classification solutions. These solutions lack flexibility, have high maintenance costs, and are difficult to adapt to the diversity and variability of financial forms, resulting in failure to meet the needs of financial form processing. Therefore, it is necessary to provide a method, system, device, and medium for recognizing financial forms. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a method, system, device and medium for identifying financial forms, which improves the problem of low accuracy in extracting nested named entities and identifying their semantic relationships from complex and changeable financial forms in the prior art.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for recognizing a financial form, comprising: obtaining a financial form to be recognized; extracting feature maps of different scales from the financial form and fusing them according to a segmentation-based text detection algorithm, and recognizing and extracting a text area in the financial form from the fused feature maps; extracting text sequence features in the text area, and transcribing the extracted text sequence features based on an attention mechanism to extract a text sequence in the text area; encoding and semantically enhancing the text sequence to obtain a comprehensive vector sequence, and decoding the comprehensive vector sequence to recognize named entities in the financial form.

[0006] In one embodiment of the present invention, the method extracts feature maps of different scales from the financial form and performs fusion processing based on the segmentation-based text detection algorithm, and identifies and extracts the text area in the financial form from the fused feature map, including: inputting the financial form into a residual network, performing feature processing through each convolutional layer group, and generating feature maps of different scales; wherein the residual network includes a plurality of cascaded convolutional layer groups, and each convolutional layer group generates a feature map of one scale; performing feature enhancement on the feature map of the minimum scale, and arranging the enhanced feature map of the minimum scale and the remaining feature maps of each scale in descending order according to the scale; based on the feature pyramid algorithm, according to the arrangement order of each feature map, each feature map is fused to obtain a fused feature map; based on the differential binarization text detection algorithm, the text area in the financial form is identified and extracted from the fused feature map.

[0007] In one embodiment of the present invention, the convolution at the preset position in the residual network is a deformable convolution.

[0008] In one embodiment of the present invention, for the first convolutional layer group of the residual network, the process of generating a feature map of a corresponding scale includes: inputting the financial form into the convolution module of the first convolutional layer group to obtain an initial feature map; inputting the initial feature map into the channel attention module of the first convolutional layer group, performing global maximum pooling and global average pooling on the financial form, and processing the two pooled feature maps based on a shared multi-layer perceptron to obtain a channel-weighted feature map; inputting the channel-weighted feature map into the spatial attention module of the first convolutional layer group, performing global maximum pooling and global average pooling on the channel-weighted feature maps, and splicing them, performing dilated convolution on the spliced ​​feature maps, and fusing the dilated convolution feature maps with the channel-weighted feature maps to obtain spatially weighted feature maps; inputting the spatially weighted feature maps into the residual module of the first convolutional layer group to generate feature maps of corresponding scales.

[0009] In one embodiment of the present invention, for each remaining convolutional layer group of the residual network, the process of generating a feature map of the corresponding scale includes: inputting the feature map generated by the previous convolutional layer group into the convolution module of the current convolutional layer group to obtain an initial feature map of the corresponding scale of the current convolutional layer group; inputting the initial feature map into the residual module of the current convolutional layer group to generate a feature map of the corresponding scale.

[0010] In one embodiment of the present invention, the text sequence features in the text area are extracted, and the extracted text sequence features are transcribed based on an attention mechanism to extract the text sequence in the text area, including: extracting the features of the text area based on a convolutional neural network, and transforming the extracted features based on context information according to a bidirectional long short-term memory network to obtain a feature sequence; based on a bidirectional long short-term memory network and an attention mechanism, character transcribing the feature sequence to extract the text sequence in the text area.

[0011] In one embodiment of the present invention, encoding and semantically enhancing the text sequence to obtain a comprehensive vector sequence, decoding the comprehensive vector sequence, and identifying the named entities in the financial form include: inputting the text sequence into a pre-trained language model to obtain an embedded vector sequence; based on a bidirectional long short-term memory network, capturing the context features of the text sequence, encoding the context features, and obtaining a context vector sequence; based on cosine similarity, screening out similar vector combinations of each context vector in the context vector sequence from a word vector library; according to an attention mechanism, weighting the similar vectors in the similar vector combination to obtain a semantically enhanced vector for each context vector, forming a semantically enhanced vector sequence of the context vector sequence; splicing the semantically enhanced vector sequence, the context vector sequence, and the embedded vector sequence to obtain a comprehensive vector sequence; based on a multi-layer entity recognizer, decoding the comprehensive vector sequence to identify and extract mutually nested named entities in the financial form; wherein each layer of the entity recognizer includes a bidirectional long short-term memory network and a dual-core convolutional neural network.

[0012] In one embodiment of the present invention, the named entities include semantic labels and entity content. After encoding and semantically enhancing the text sequence to obtain a comprehensive vector sequence, decoding the comprehensive vector sequence, and identifying the named entities in the financial form, the method further includes: inputting each named entity into a feedforward neural network for feature extraction to generate semantic label features and content features corresponding to each named entity; combining the semantic label features and entity content features of each named entity to obtain entity pair features of each named entity; inputting the entity pair features of each named entity into a dual affine classifier to obtain a relationship score between each named entity; for each named entity: selecting another named entity with the highest relationship score as the relationship named entity of the current named entity to form a relationship entity pair.

[0013] In one embodiment of the present invention, a financial form recognition system is also provided, the system comprising: a form acquisition module, used to acquire a financial form to be recognized; a text detection module, used to extract feature maps of different scales from the financial form and perform fusion processing based on a segmentation-based text detection algorithm, and recognize and extract the text area in the financial form from the fused feature map; a text recognition module, used to extract text sequence features in the text area, and transcribe the extracted text sequence features based on an attention mechanism to extract the text sequence in the text area; a financial nested named entity recognition module, used to encode and semantically enhance the text sequence to obtain a comprehensive vector sequence, and decode the comprehensive vector sequence to recognize the named entities in the financial form.

[0014] In one embodiment of the present invention, an electronic device is also provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements any of the financial form recognition methods described above.

[0015] In one embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes any of the above-mentioned financial form recognition methods.

[0016] As described above, a method, system, device and medium for identifying financial forms of the present invention have the following beneficial effects: after obtaining the financial form to be identified, a segmentation-based text detection algorithm is used to extract and fuse multi-scale feature maps, thereby improving the detection accuracy of text areas of different scales and complex backgrounds. The text sequence features of the text area are extracted, and the transcription process is combined with the attention mechanism to enhance the recognition accuracy of the text content. The text sequence is encoded and semantically enhanced to obtain a comprehensive vector sequence, which is then decoded to successfully identify the named entities in the financial form. The present invention realizes the automated, refined extraction and structured processing of financial form information. For images of complex financial forms, text recognition and extraction are performed, and key named entities and their relationships are extracted to form a unified structured output. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of a method for identifying a financial form provided by an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of a financial form;

[0019] Figure 3 A network structure diagram for extracting text areas from financial forms in the present invention;

[0020] Figure 4 Generate a network structure diagram of the text area for the first convolutional layer group and the remaining convolutional layer group of the present invention;

[0021] Figure 5 A network structure diagram for extracting text sequences from text regions in the present invention;

[0022] Figure 6 A network structure diagram for extracting a named entity set from a text sequence in the present invention;

[0023] Figure 7 A network structure diagram of the relationship between entities extracted from a named entity set by the present invention;

[0024] Figure 8 Shown is a structural block diagram of a financial form recognition system provided by an embodiment of the present invention;

[0025] Fig. 9 Shown is a structural schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0027] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0028] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0029] The inventors found that for the form technology in the financial field, facing complex transaction data and customer information, the traditional manual processing method can no longer meet the growing demand. Therefore, it is particularly important to use deep learning technology to realize automated form processing. For the visual solution of financial forms, it has higher requirements than general text recognition tasks. The data features in financial forms have higher recognition, such as numbers, dates, barcodes and other special elements need to be accurately extracted and recognized. In addition, the environmental conditions of financial forms are poor, including dark light and many influencing factors. Therefore, in low illumination and complex environments, the use of deep learning technology to achieve panoramic image acquisition and feature point recognition of financial forms has better effects, can effectively record abnormal situations and provide more reliable data support. Traditional template matching schemes require templates to be compiled in advance, and the recognition effect is not good for complex form structures and data formats with large changes. Rule-based text extraction schemes rely on manually written rules. For specific form types and new form styles, the rules need to be constantly adjusted and updated, and the maintenance cost is high. Hybrid feature classification schemes usually require a large amount of labeled data and feature engineering, which is difficult to cope with the challenges of diversity and variability of financial forms. The generated data has low practical value for risk management and security of financial institutions. The problem of low accuracy of existing technologies in extracting nested named entities and identifying their semantic relationships from complex and changeable financial forms has been improved.

[0030] The present invention provides a method for identifying financial forms. After obtaining the financial form to be identified, a segmentation-based text detection algorithm is used to extract and fuse multi-scale feature maps, thereby improving the detection accuracy of text areas of different scales and complex backgrounds. The text sequence features of the text area are extracted, and transcription processing is performed in combination with the attention mechanism, thereby enhancing the recognition accuracy of the text content. The text sequence is encoded and semantically enhanced to obtain a comprehensive vector sequence, which is then decoded to successfully identify the named entities in the financial form. The present invention realizes the automated, refined extraction and structured processing of financial form information. For images of complex financial forms, text recognition and extraction are performed, and key named entities and their relationships are extracted to form a unified structured output.

[0031] See also Figure 1 , the identification method of the financial form includes the following steps:

[0032] S1. Obtain the financial form to be identified.

[0033] Financial forms refer to structured or semi-structured documents used to record and transmit information related to financial business. Common types of financial forms include, but are not limited to, bank account application forms, loan application forms, insurance application forms, financial statements, and stock transaction forms. It should be noted that the method described in the present invention can be applied to the recognition and association of the content of printed financial forms, as well as the recognition and association of the content of handwritten financial forms. For handwritten financial forms, an image of the financial form to be identified can be obtained by taking a photo. For printed financial forms, an image of the financial form to be identified can be obtained by scanning or uploading an electronic document. Figure 2 As shown in FIG. 1 , an image of a handwritten financial form (i.e., a loan application form) is shown. Since handwriting recognition is difficult, relying solely on manual recognition statistics is time-consuming and labor-intensive. Therefore, this method can be used to identify the relationship between various named entities. For example, from Figure 2 One of the relationships that can be identified is: Contact name: Zhang San, and his ID type: ID card.

[0034] S2. According to a segmentation-based text detection algorithm, feature maps of different scales are extracted from the financial form and fused, and the text area in the financial form is identified and extracted from the fused feature maps.

[0035] The present invention adopts the segmentation-based text detection (DBNet) algorithm, and performs multi-level convolution on the financial form through multiple cascaded convolution groups to extract feature maps of different scales, and gradually fuses these feature maps of different scales in a top-down manner through feature fusion to enhance the overall feature expression ability. Since the fused feature map contains local and global information, the text area identified from it will be more accurate, so as to improve the accuracy of subsequent text recognition.

[0036] In one embodiment of the present invention, extracting feature maps of different scales from the financial form and fusing them according to a segmentation-based text detection algorithm, and identifying and extracting text areas in the financial form from the fused feature maps include:

[0037] Inputting the financial form into a residual network, performing feature processing through each convolutional layer group, and generating feature maps of different scales; wherein the residual network includes a plurality of cascaded convolutional layer groups, and each convolutional layer group generates a feature map of one scale;

[0038] Perform feature enhancement on the feature map of the smallest scale, and arrange the enhanced feature map of the smallest scale and the feature maps of the remaining scales in descending order of scale;

[0039] Based on the feature pyramid algorithm, each feature map is fused according to the arrangement order of each feature map to obtain a fused feature map;

[0040] Based on the differential binarization text detection algorithm, the text area in the financial form is identified and extracted from the fused feature map.

[0041] like Figure 3 As shown, the financial form image is input into a residual network (such as ResNet50). Since the residual network is composed of multiple cascaded convolutional layer groups, each convolutional layer group performs feature processing on the input information and extracts features therein, thereby generating a feature map of a preset scale. Furthermore, in order to further improve the detection and recognition capabilities of text areas, the convolutions at preset positions in the residual network are deformable convolutions. Specifically, the present invention also replaces all 3×3 convolutional layers of the Conv3, Conv4, and Conv5 layers in the residual network with deformable convolutions. Since the deformable convolution can adaptively adjust the sampling position of the convolution kernel, the detected text area can be located more accurately to improve the flexibility of feature extraction. Through the above operations, a set of feature maps of different scales output by the residual network (such as Figure 3 {C2, C3, C4, C5} in ), each feature map corresponds to a scale. Through this multi-scale feature extraction, text regions of different sizes and shapes in financial form images can be better captured to improve the accuracy of subsequent text recognition. In addition, in order to enrich the semantic information of high-level features to enhance the network's perception of complex text regions and improve the accuracy of text region detection, the present invention also adds a feature enhancement algorithm (FFA) to the feature map of the smallest scale for feature enhancement.

[0042] Among them, the smallest scale feature map is the feature map obtained by the highest layer of the residual network (such as Figure 3 C5 in the above). Specifically, firstly, the minimum-scale feature map generated by the convolution layer group is subjected to channel dimensionality reduction processing. By performing 1×1 convolution on the feature map, the number of channels of the feature map is reduced from high dimension to 256 dimensions, thereby greatly reducing the amount of calculation. The feature map after dimensionality reduction is subjected to random pooling processing to obtain multiple pooled feature maps. Specifically, three feature maps {α1=0.1, α2-0.2, α3=0.3} of different scales and containing rich contextual semantic information can be formed. For each of the above pooled feature maps, bilinear upsampling processing is performed to restore its size to the same size as the original feature map (i.e., C5). The feature fusion of each feature map after upsampling is performed, and the sigmoid activation function is used to obtain an enhanced minimum-scale feature map. As shown in formula (1):

[0043]

[0044] Among them, Pool(C5,α i ) represents the α i times the downsampling operation, F 1×1 represents 1*1 convolution, UP(·,α i ) represents the feature map α i The bilinear upsampling is times, and FAA (C5) represents the enhanced minimum scale feature map. The enhanced minimum scale feature map and the remaining feature maps of each scale {C2, C3, C4} are arranged in descending order of scale (from large to small), and the order is {C5′, C4, C3, C2}.

[0045] Then, based on the feature pyramid network, starting from the smallest scale feature map, the feature maps are upsampled layer by layer to match their size with the next layer of high-resolution feature maps. At each scale, the upsampled feature map Mi+1 is horizontally connected with the corresponding high-resolution feature map Mi (i.e., the two feature maps are added element by element), fusing high-level semantic information and low-level spatial detail information. Then, the feature maps after each horizontal connection are fused, and in this way, a fused feature map is finally obtained.

[0046] The fused feature map is input into the DB text detection algorithm to predict and generate a probability map and a threshold map. The probability map and the threshold map are converted into a binary map through a differentiable binarization function, thereby accurately segmenting the text area. Thus, the text area in the financial form is extracted based on the binarization map, and a text detection result prediction box is generated.

[0047] For the first convolutional layer group of the residual network, the process of generating a feature map of the corresponding scale includes:

[0048] Inputting the financial form into the convolution module of the first convolution layer group to obtain an initial feature map;

[0049] Inputting the initial feature map into the channel attention module of the first convolutional layer group, performing global maximum pooling and global average pooling on the financial form respectively, and processing the two pooled feature maps based on a shared multi-layer perceptron to obtain a channel weighted feature map;

[0050] Inputting the channel weighted feature map into the spatial attention module of the first convolutional layer group, performing global maximum pooling and global average pooling on the channel weighted feature map respectively and splicing them, performing dilated convolution on the spliced ​​feature map, and fusing the feature map after dilated convolution with the channel weighted feature map to obtain a spatial weighted feature map;

[0051] The spatially weighted feature map is input into the residual module of the first convolutional layer group to generate a feature map of the corresponding scale.

[0052] The financial form image is input into the first convolution layer group of the residual network. Through the convolution operation, the initial feature map is obtained.

[0053] In one embodiment of the present invention, for each remaining convolutional layer group of the residual network, the process of generating a feature map of a corresponding scale includes:

[0054] Input the feature map generated by the previous convolutional layer group into the convolution module of the current convolutional layer group to obtain the initial feature map of the corresponding scale of the current convolutional layer group;

[0055] The initial feature map is input into the residual module of the first convolutional layer group to generate a feature map of the corresponding scale.

[0056] like Figure 4 As shown, the first convolutional layer group includes a convolutional module, a channel attention module, a spatial attention module and a residual module that are cascaded in sequence. The financial form is input into the convolutional module of the first convolutional layer group of the residual network (such as ResNet50). After the initial convolution operation, the initial feature map F (size is H*W*C) is obtained. The initial feature map F is input into the channel attention module of the first convolutional layer group. First, the initial feature map F is subjected to global average pooling and global maximum pooling respectively to obtain two feature vectors of size 1×1×C. These two feature vectors are respectively input into a shared multi-layer perceptron (MLP). The MLP consists of a hidden layer to reduce the amount of parameter calculation. The activation size of the hidden layer is set to R / C=r×1×1, where R is the drop rate. After the two feature vectors output by the MLP are added, the channel attention map M is generated through the sigmoid activation function. c (F). The processing process of the channel attention module is shown in formula (2):

[0057] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (2)

[0058] Among them, σ is the sigmoid function, AvgPool(F) represents the global average pooling of the initial feature map F, MaxPool(F) represents the global maximum pooling of the initial feature map F, MLP is a multi-layer perceptron, M c (F) is the channel attention map generated by the multi-layer perceptron. c (F) is element-wise multiplied with the input initial feature map F to obtain the channel weighted feature map output by the channel attention module. In this way, important channel features can be focused on and unimportant channel features can be suppressed, thereby providing a more powerful feature representation for the subsequent spatial attention module.

[0059] The channel weighted feature map output by the channel attention module is input into the spatial attention module. First, the channel weighted feature map is again subjected to global average pooling and global maximum pooling in the channel dimension to obtain two feature maps of size H×W×1. The two feature maps are spliced ​​in the channel dimension to form a feature map of size H×W×2. In order to obtain stronger spatial position information and reduce the roughness problem of text positioning, the original conventional convolution in the spatial attention module is replaced with a hole convolution, and the spliced ​​feature map is convoluted. The present invention takes into account the advantages of hole convolution over conventional convolution: 1. Inserting blank areas between convolution kernels can integrate information in a larger range, so it can more effectively suppress noise interference and improve the expression ability of spatial features. 2. Conventional convolution may cause information loss when smoothing small areas, especially in edge and detail processing. 3. Hollow convolution helps to retain more detail features, thereby reducing the roughness of text positioning. The use of hole convolution can avoid the decrease in feature map size caused by downsampling to a certain extent, thereby maintaining high-resolution information. This is particularly important for pixel-level accuracy in positioning tasks. The processing process of the spatial attention module is shown in formula (3):

[0060]

[0061] in, is a dilated convolution operation with a convolution kernel of 7×7, σ is a sigmoid function, AvgPool(M c (F)) represents the channel weighted feature map M c (F) Perform global average pooling, MaxPool (M c (F)) represents the channel weighted feature map M c (F) Perform global maximum pooling, M s (F) is the spatial attention map. The generated spatial attention map M s (F) Channel weighted feature map M input to the spatial attention module c (F) Perform element-by-element multiplication to obtain the spatially weighted feature map generated by the spatial attention module. The spatially weighted feature map is input into the residual module of the first convolutional layer group for further feature extraction and nonlinear transformation, and outputs a feature map of the scale corresponding to the first convolutional layer group. The residual module uses skip connections to alleviate the gradient vanishing problem in deep networks and ensure the effective transfer of features.

[0062] In one embodiment of the present invention, for each remaining convolutional layer group of the residual network, the process of generating a feature map of a corresponding scale includes:

[0063] Input the feature map generated by the previous convolutional layer group into the convolution module of the current convolutional layer group to obtain the initial feature map of the corresponding scale of the current convolutional layer group;

[0064] The initial feature map is input into the residual module of the current convolutional layer group to generate a feature map of the corresponding scale.

[0065] For each remaining convolutional layer group in the residual network (i.e., subsequent layer groups except the first convolutional layer group), the process of generating the corresponding scale feature map is as follows: the feature map generated by the previous convolutional layer group is input into the convolution module of the current convolutional layer group, and the feature map is further convolved to extract deeper features to obtain the initial feature map of the corresponding scale of the current convolutional layer group. The initial feature map obtained by the current convolutional module is input into the residual module of the convolutional layer group, and the gradient disappearance phenomenon in the deep network is greatly alleviated through jump connections. The corresponding scale feature map obtained can retain important feature content.

[0066] S3. Extract text sequence features in the text area, and transcribe the extracted text sequence features based on an attention mechanism to extract the text sequence in the text area.

[0067] Text sequence features refer to feature representations extracted from text regions, including but not limited to information such as character shape, strokes, and arrangement that can identify specific text content. Text sequence refers to readable text that is ultimately extracted after transcription processing.

[0068] Specifically, in one embodiment of the present invention, extracting text sequence features in the text region, and transcribing the extracted text sequence features based on an attention mechanism to extract the text sequence in the text region includes:

[0069] Extracting features of the text region based on a convolutional neural network, and transforming the extracted features according to context information based on a bidirectional long short-term memory network to obtain a feature sequence;

[0070] Based on a bidirectional long short-term memory network and an attention mechanism, the feature sequence is transcribed into characters to extract a text sequence in the text area.

[0071] The present invention converts the text area into a text sequence through a text recognition algorithm model, and the text recognition algorithm model includes an encoder and a decoder. Among them, the task of the encoder is to convert the input image into a text feature sequence via a convolutional neural network and a bidirectional long short-term memory network, and the task of the decoder is to generate text from the encoded text feature sequence through a character transcription method, obtain the final text recognition result and output it. The text recognition algorithm model uses the ResNet34 deep residual network as the backbone network for feature extraction. In order to reduce the number of parameters, the fully connected layer in the residual network is removed, and only the convolution layer and the pooling layer are retained. In the feature extraction process, the text features are processed using a bidirectional long short-term memory network to generate a feature sequence. Furthermore, the text recognition algorithm model also introduces an attention mechanism module and a bidirectional decoder structure to improve the accuracy of text recognition.

[0072] Specifically, Figure 5 As shown in the figure, the encoder ResNet34 is a recurrent neural network based on convolution. First, the input text area image is input into the ResNet34 convolutional neural network, and feature extraction is performed in the convolution layer to obtain the convolution feature map of the text area image, where the size of the feature map is w×h×d, w represents width, h represents height, and d represents the number of channels. Then, the height h of the feature map is reduced to 1 through the pooling layer, and a feature map of size w×1×d is obtained. The reduced feature map is expanded in the width direction to form a feature sequence f=(f1,f2,…,f w ), where each f t is the feature vector. The feature sequence is processed using a bidirectional long short-term memory network (BiLSTM) to capture the contextual information of the characters in the sequence and generate a more semantic feature sequence. Finally, the feature map is converted into a feature sequence through a bidirectional long short-term memory network. For each time step t in the feature sequence, the label prediction probability is calculated as shown in formula (4):

[0073] y t =softmax(Mf t +b),t=1,…,w (4)

[0074] Among them, y t is the output result at time step t, M is the trained weight matrix, f t is the feature vector at time step t, b is the bias vector, and w is the length of the feature sequence.

[0075] The decoder is a BiLSTM based on bidirectional attention, capturing complementary semantic features from two directions. The decoder outputs a text sequence of length T, which is t-1 and the predicted value y at the previous time step t-1, predict the output of the current time step t. The decoder first calculates the attention score e of the current time step t t,i , as shown in formula (5):

[0076] e t,i =w T tanh(Ws t-1 +Vh i +b) (5)

[0077] Among them, e t,i is the decoder hidden state s at the current time step t t-1 With the encoder hidden state h i The attention score between t-1 is the hidden state of the decoder at time step t-1, h i is the hidden state of the encoder at position i, w, W and V are all weight parameters that need to be trained, and b is the bias. According to the attention score e t,i The attention weight α of the current time step t can be obtained t.i , as shown in formula (6):

[0078]

[0079] Among them, α t.i is the attention weight of encoder position i at time step t, and n is the length of the sequence output by the encoder. Attention weight α t.i It can effectively represent the importance of each item of the encoder output. Use the attention weight to weight the hidden state of the encoder to obtain the context vector g at time step t t , as shown in formula (7):

[0080]

[0081] The context vector g t , the hidden state s at the previous time step t-1 and the embedding vector f(y t-1 ) as input and update the decoder’s hidden state s at the current time step t t And the output feature vector x t , as shown in formula (8):

[0082] (x t ,s t )=BiSLTM(s t-1 ,[g t ,f(y t-1 )]) (8)

[0083] Among them, [g t ,f(yt-1 )] means concatenating the context vector and the embedding vector, f(y t-1 ) represents the embedding vector of the predicted output of the previous time step. As shown in formula (9), the output feature vector x of the decoder is used t Calculate the output probability distribution p(y t ):

[0084] p(y t )=softmax(W0x t +b0) (9)

[0085] Among them, W0 and b0 are the weight matrix and bias vector of the decoder output layer respectively. According to the output probability distribution, the category with the highest probability is selected to determine the predicted output y at the current time step t t , as shown in formula (10):

[0086] y t = argmaxp(y t ) (10)

[0087] It should be noted that the convergence of the loss function plays a decisive role in the final text recognition model structure. The loss function of the bidirectional decoder is selected on the loss function to constrain the final model, improve the running speed of the model and the convergence efficiency of the network structure, and thus improve the text recognition effect of the model.

[0088] In the prediction process, the bidirectional decoder of the present invention adopts the beam search method to obtain the final decoding result from two directions, and then selects the result with a higher score as the output. The loss function L is shown in formula (11):

[0089]

[0090] where p l represents the probability density of the left-to-right decoder, p r Represents the probability density of the decoder from right to left. After training, in the inference phase of the model, the beam search method is still used to select the one with the higher score as the final decoding result according to the decoding results corresponding to the two directions of the decoder.

[0091] S4. Encode and semantically enhance the text sequence to obtain a comprehensive vector sequence, and decode the comprehensive vector sequence to identify named entities in the financial form.

[0092] Specifically, in one embodiment of the present invention, encoding and semantically enhancing the text sequence to obtain a comprehensive vector sequence, and decoding the comprehensive vector sequence to identify the named entities in the financial form include:

[0093] Inputting the text sequence into a pre-trained language model to obtain an embedding vector sequence;

[0094] Based on a bidirectional long short-term memory network, capturing context features of the text sequence, encoding the context features, and obtaining a context vector sequence;

[0095] Based on the cosine similarity, a similar vector combination of each context vector in the context vector sequence is selected from the word vector library;

[0096] According to the attention mechanism, similar vectors in the similar vector combination are weighted to obtain a semantic enhancement vector for each context vector, thereby forming a semantic enhancement vector sequence of the context vector sequence;

[0097] Concatenate the semantic enhancement vector sequence, the context vector sequence, and the embedding vector sequence to obtain a comprehensive vector sequence;

[0098] Based on a multi-layer entity recognizer, the comprehensive vector sequence is decoded to identify and extract mutually nested named entities in the financial form; wherein each layer of entity recognizer includes a bidirectional long short-term memory network and a dual-core convolutional neural network.

[0099] like Figure 6 As shown in the figure, a large number of nested entities with complex associations and structural characteristics in the financial corpus are identified and processed, so as to achieve the purpose of obtaining nested entities according to the text sequence. First, the input recognized text sequence is converted into a distributed vector representation to obtain an embedded vector sequence. And through the bidirectional long short-term memory network, by extracting text features, the context vector sequence corresponding to the embedded vector sequence is obtained. Then, based on cosine similarity, several similar vectors of each context vector are found from the pre-trained word vector library (such as Tencent embedding). The Attention mechanism is added to give more attention to the more important similar vectors in the current context to obtain the semantic enhancement vector. The weights of the embedding vector and the semantic enhancement vector are rebalanced to form the final semantic enhancement vector sequence. Then the semantic enhancement vector sequence, the context vector sequence and the embedding vector sequence of the pre-trained language model are spliced ​​to obtain a comprehensive vector sequence. Finally, the comprehensive vector sequence is input into the multi-layer entity recognizer for decoding processing. Each layer of the entity recognizer includes a bidirectional long short-term memory network (BiLSTM) and a dual-core convolutional neural network (CNN), which realizes the recognition of entities of different lengths layer by layer. Finally, a set of financial named entities is obtained.

[0100] Specifically, the text sequence C = {c1, c2, c3, ..., c i} is input into a pre-trained language model (such as BERT), where c i Represents the i-th character of the original input text sequence. Based on character-level word embedding, find the pre-trained vector corresponding to each character from the pre-trained word vector library (such as TencentEmbedding) to form an embedded vector sequence V = {v1, v2, v3, ... v i}. For characters for which no corresponding pre-trained vector is found, they will be initialized to a random vector. i Represents the i-th character c i The corresponding embedding vector. The embedding vector sequence V is input into the encoding layer (i.e., the bidirectional long short-term memory network BiLSTM) to learn the contextual features of the text, and the BiLSTM is used to extract the text features to obtain the context vector sequence As shown in formula (12):

[0101]

[0102] Among them, BiLSTM is the encoding layer, Embed is the embedding layer, is the context vector sequence, and C is the text sequence. The present invention does not directly perform decoding after encoding, but uses a semantic enhancement algorithm to semantically enhance the context vector sequence between character encoding and decoding. Based on cosine similarity, find the n closest vectors from the pre-trained word vector library to form a similar vector combination V of the i-th context vector i , as shown in formula (13):

[0103] V i = {v i1 ,v i2 ,v i3 ...v ij ...v in} (13)

[0104] Among them, v ij v i A similar vector of . Using the attention mechanism, the weight of the similar vector is calculated to highlight the important similar vectors and complete the data enhancement processing. Therefore, for each similar word, their weight can be calculated by formula (14):

[0105]

[0106] in, is the latent vector (i.e., context vector) of the input sequence after context encoding at the encoding layer, v i,jis the similarity vector of the i-th context vector. Then, the weighted semantic enhancement vector x of the i-th embedding vector is calculated using formula (15): i :

[0107]

[0108] Considering that in different contexts, similarity vectors and original character vectors (i.e., embedded vectors) will have different contributions, it is necessary to control the flow of information through the reset gate and measure their contributions, as shown in formula (16):

[0109] g=σ(R1·h i +R2·x i +b) (16)

[0110] Among them, R1 and R2 are trainable matrices, b is the corresponding bias, σ is the sigmoid function, and g is the reset gate. Then, the original context vector and the corresponding semantic enhancement vector are fused through formula (17) to obtain the fused vector:

[0111]

[0112] Among them, i is the final semantic enhancement vector after fusion (i.e., the final semantic enhancement vector of the i-th embedding vector), is the context vector of the i-th embedding vector, is element-wise multiplication, is vector concatenation, [·] indicates that the result is a vector. Each semantic enhancement vector is arranged in sequence to obtain a semantic enhancement vector sequence o.

[0113] Furthermore, in order to improve the performance of the model, the present invention converts the embedding vector V of the pre-trained language model into {v1, v2, v3, ... v i}, semantic enhancement vector o, and context vector sequence output by BiLSTM After concatenation, the dimension is reduced through a fully connected layer to obtain the comprehensive vector sequence shown in formula (18):

[0114]

[0115] Among them, Linear is a fully connected layer. Then use the decoder to decode the above comprehensive vector sequence Decoding is performed. The decoder consists of L layers of entity recognizers, each of which consists of a BiLSTM and a dual-core CNN to recognize all entities with an entity span of [1, L]. CNN aggregates hidden states at different levels, allowing the decoding layer to capture low-level information to recognize short entities and high-level information to recognize long entities, helping to alleviate the problem of error propagation and improve the accuracy of named entity recognition. The input of the first layer of the decoding layer is different from that of other layers and has not been convolved by the CNN. In order to avoid the impact of this input difference on the model, the embedding is layer-normalized before entering the LSTM, as shown in formulas (19) and (20):

[0116]

[0117]

[0118] in, is the input of the lth layer, BiLSTM is the decoding layer, LayerNorm is the layer normalization, and h i is the hidden state of the i-th position in the l-th layer of the decoding layer. The final output result is an entity label set ε, for ε, as shown in formula (20):

[0119] ε={(E0,c0),(E1,c1),...,(E3,c n )} (20)

[0120] Where C = {c0, c1, ..., c n} is the extracted named entity set, E={E0,E1,...,E n} is the semantic label corresponding to the named entity (i.e., the weight of the entity content). Through the above process, nested financial named entities can be identified and extracted from the text sequence to meet the needs of complex financial text information extraction.

[0121] In an embodiment of the present invention, the named entity includes a semantic tag and entity content. After encoding and semantically enhancing the text sequence to obtain a comprehensive vector sequence, decoding the comprehensive vector sequence, and identifying the named entity in the financial form, the method further includes:

[0122] Extract the semantic label features and content features corresponding to each named entity;

[0123] The semantic label features and entity content features of each named entity are combined to obtain the entity pair features of each named entity;

[0124] Input the entity pair features of each named entity into the feedforward neural network to obtain the question entity features and the answer entity features accordingly;

[0125] Input the question entity features and answer entity features of each named entity into a bi-affine classifier to obtain the relationship scores between each named entity;

[0126] For each named entity: select another named entity with the highest relationship score as the relationship named entity of the current named entity to form a relationship entity pair.

[0127] like Figure 7 As shown in the figure, for the judgment question pairs that only appear in financial forms, a pipeline scheme is used for relationship extraction based on the named entity set ε output in the financial nested named entity recognition module. By processing the input entity set ε containing semantic labels and entity content, the relationship extraction model is used to complete the entity relationship extraction and matching of the questions and their answers in the text, and finally a set of matched question pairs is obtained. Specifically, the relationship between each entity extracted from the named entity is obtained through the relationship extraction model, and the relationship extraction model includes at least a feedforward neural network and a biaffine classifier. When training the relationship extraction model, for the entity set ∈ that has been obtained, define R = {r0, r1, …, r m} as a set of semantic relationship labels between named entities. In order to implement the relationship extraction function F RE :(ε,R)→ρ, that is, the relation extraction function F RE Receive the named entity set ε and the relationship label set R, and output the entity relationship set ρ. ρ is shown in formula (21):

[0128] ρ={(question0,answer0,r0),...,(question n ,answer n ,r n )} (twenty one)

[0129] in, answer i Respectively indicate that the two named entities correspond to the question and answer in the document. In this way, the relationship extraction and matching of named entities and corresponding relationship pairs are completed. In the model training phase, when constructing the data set, the semantic relationship label set that may be generated is also manually constructed to complete the establishment of the relationship candidate set of all semantic entities. The candidate set R contains all possible relationships between entities, such as "question-answer", "definition-term", etc.

[0130] Whether in the model training or reasoning phase, for each pair of entities obtained by the above relationship extraction function The semantic label features are obtained through a specific type of embedding layer. Each entity's semantic label corresponds to an embedding vector. iInput two layers of feedforward neural network (FFNN), namely FFNN S and FFNN E , complete the feature extraction of question and answer entities, as shown in formula (22)(23):

[0131]

[0132] Among them, h s (i) h e (i) are question entity features and answer entity features, is the feature vector of the problem entity, is the feature vector of the answer entity. In the relation extraction process, these two feature vectors are used in a bi-affine classifier to calculate the relationship score r between the question entity and the answer entity. m (i), as shown in formula (24):

[0133]

[0134] Among them, U m is the bi-affine transformation matrix, W m and b m are the weight matrix and bias vector of affine transformation respectively, By calculating the relationship score, the relationship type between each question-answer entity pair can be predicted, thereby completing the relationship extraction task.

[0135] like Figure 2 As shown in the figure, for a handwritten or printed financial form, the text sequence is first identified, and through nested named entity recognition, a named entity set ∈ is obtained: {(entity category E1: applicant name, entity content C1: Zhang San), (entity category E2: ID number, entity content C2: 11234177001251112)}, etc. Through relationship extraction, it can be known that the relationship entity pair set is: (applicant name, Zhang San) and (ID number, 11234177001251112) have an identity association relationship.

[0136] See also Figure 8The financial form recognition system 100 includes: a form acquisition module 110, which is used to acquire a financial form to be recognized; a text detection module 120, which is used to extract feature maps of different scales from the financial form and perform fusion processing based on a segmentation-based text detection algorithm, and recognize and extract the text area in the financial form from the fused feature map; a text recognition module 130, which is used to extract text sequence features in the text area, and transcribe the extracted text sequence features based on an attention mechanism to extract the text sequence in the text area; a financial nested named entity recognition module 140, which is used to encode and semantically enhance the text sequence to obtain a comprehensive vector sequence, and decode the comprehensive vector sequence to recognize the named entities in the financial form.

[0137] The specific definition of the financial form recognition system can be found in the definition of the financial form recognition method above, which will not be repeated here. Each module in the above financial form recognition system can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware format, or can be stored in the memory of the computer device in software format, so that the processor can call the corresponding operations of each of the above modules.

[0138] It should be noted that, in order to highlight the innovative part of the present invention, the present embodiment does not introduce modules that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other modules in the present embodiment.

[0139] See also Fig. 9 The electronic device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a financial form recognition program.

[0140] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 12 can be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 12 can also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 12 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as the identification code of the financial form, etc., but also can be used to temporarily store data that has been output or is to be output.

[0141] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect various components of the entire electronic device 1, and executes or executes programs or modules (such as financial form recognition programs, etc.) stored in the memory 12, and calls data stored in the memory 12 to execute various functions of the electronic device 1 and process data.

[0142] The processor 13 executes the operating system and various installed applications of the electronic device 1. The processor 13 executes the applications to implement the steps in the above-mentioned financial form recognition method.

[0143] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a form acquisition module 110, a text detection module 120, a text recognition module 130, and a financial nested named body recognition module 140.

[0144] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, and the computer-readable storage medium can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the financial form recognition method described in various embodiments of the present application.

[0145] In summary, the present invention discloses a method, system, device and medium for identifying financial forms. After obtaining the financial form to be identified, a segmentation-based text detection algorithm is used to extract and fuse multi-scale feature maps, thereby improving the detection accuracy of text areas of different scales and complex backgrounds. The text sequence features of the text area are extracted, and the transcription process is combined with the attention mechanism to enhance the recognition accuracy of the text content. The text sequence is encoded and semantically enhanced to obtain a comprehensive vector sequence, which is then decoded to successfully identify the named entities in the financial form. The present invention realizes the automated, refined extraction and structured processing of financial form information. For images of complex financial forms, text recognition and extraction are performed, and key named entities and their relationships are extracted to form a unified structured output. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0146] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for identifying a financial form, characterized in that: The method comprises: Get the financial form to be identified; Extracting feature maps of different scales from the financial form and fusing them according to a segmentation-based text detection algorithm, and identifying and extracting text areas in the financial form from the fused feature maps; Extracting text sequence features in the text region, and performing transcription processing on the extracted text sequence features based on an attention mechanism to extract a text sequence in the text region; The text sequence is encoded and semantically enhanced to obtain a comprehensive vector sequence, and the comprehensive vector sequence is decoded to identify the named entities in the financial form.

2. The method for identifying a financial form according to claim 1, characterized in that: The method of extracting feature maps of different scales from the financial form and fusing them according to the segmentation-based text detection algorithm, and identifying and extracting the text area in the financial form from the fused feature maps, includes: Inputting the financial form into a residual network, performing feature processing through each convolutional layer group, and generating feature maps of different scales; wherein the residual network includes a plurality of cascaded convolutional layer groups, and each convolutional layer group generates a feature map of one scale; Perform feature enhancement on the feature map of the smallest scale, and arrange the enhanced feature map of the smallest scale and the feature maps of the remaining scales in descending order of scale; Based on the feature pyramid algorithm, each feature map is fused according to the arrangement order of each feature map to obtain a fused feature map; Based on the differential binarization text detection algorithm, the text area in the financial form is identified and extracted from the fused feature map.

3. The method for identifying a financial form according to claim 2, characterized in that: The convolution at the preset position in the residual network is a deformable convolution.

4. The method for identifying a financial form according to claim 2, characterized in that: For the first convolutional layer group of the residual network, the process of generating a feature map of the corresponding scale includes: Inputting the financial form into the convolution module of the first convolution layer group to obtain an initial feature map; Inputting the initial feature map into the channel attention module of the first convolutional layer group, performing global maximum pooling and global average pooling on the financial form respectively, and processing the two pooled feature maps based on a shared multi-layer perceptron to obtain a channel weighted feature map; Inputting the channel weighted feature map into the spatial attention module of the first convolutional layer group, performing global maximum pooling and global average pooling on the channel weighted feature map respectively and splicing them, performing dilated convolution on the spliced ​​feature map, and fusing the feature map after dilated convolution with the channel weighted feature map to obtain a spatial weighted feature map; The spatially weighted feature map is input into the residual module of the first convolutional layer group to generate a feature map of the corresponding scale.

5. The method for identifying a financial form according to claim 2, characterized in that: For each remaining convolutional layer group of the residual network, the process of generating a feature map of a corresponding scale includes: Input the feature map generated by the previous convolutional layer group into the convolution module of the current convolutional layer group to obtain the initial feature map of the corresponding scale of the current convolutional layer group; The initial feature map is input into the residual module of the current convolutional layer group to generate a feature map of the corresponding scale.

6. The method for identifying a financial form according to claim 1, characterized in that: The extracting of text sequence features in the text region and transcribing the extracted text sequence features based on an attention mechanism to extract the text sequence in the text region includes: Extracting features of the text region based on a convolutional neural network, and transforming the extracted features according to context information based on a bidirectional long short-term memory network to obtain a feature sequence; Based on a bidirectional long short-term memory network and an attention mechanism, the feature sequence is transcribed into characters to extract a text sequence in the text area.

7. The method for identifying a financial form according to claim 1, characterized in that: The encoding and semantic enhancement of the text sequence to obtain a comprehensive vector sequence, and decoding the comprehensive vector sequence to identify the named entities in the financial form include: Inputting the text sequence into a pre-trained language model to obtain an embedding vector sequence; Based on a bidirectional long short-term memory network, capturing context features of the text sequence, encoding the context features, and obtaining a context vector sequence; Based on the cosine similarity, a similar vector combination of each context vector in the context vector sequence is selected from the word vector library; According to the attention mechanism, similar vectors in the similar vector combination are weighted to obtain a semantic enhancement vector for each context vector, thereby forming a semantic enhancement vector sequence of the context vector sequence; Concatenate the semantic enhancement vector sequence, the context vector sequence, and the embedding vector sequence to obtain a comprehensive vector sequence; Based on a multi-layer entity recognizer, the comprehensive vector sequence is decoded to identify and extract mutually nested named entities in the financial form; wherein each layer of entity recognizer includes a bidirectional long short-term memory network and a dual-core convolutional neural network.

8. The method for identifying a financial form according to claim 1, characterized in that: The named entity includes a semantic tag and entity content. After encoding and semantically enhancing the text sequence to obtain a comprehensive vector sequence, decoding the comprehensive vector sequence, and identifying the named entity in the financial form, the method further includes: Each named entity is input into a feedforward neural network for feature extraction to generate semantic label features and content features corresponding to each named entity; The semantic label features and entity content features of each named entity are combined to obtain the entity pair features of each named entity; Input the entity pair features of each named entity into the bi-affine classifier to obtain the relationship score between each named entity; For each named entity: select another named entity with the highest relationship score as the relationship named entity of the current named entity to form a relationship entity pair.

9. A financial form recognition system, characterized in that: The system comprises: A form acquisition module, used to acquire the financial form to be identified; A text detection module, for extracting feature maps of different scales from the financial form and fusing them according to a segmentation-based text detection algorithm, and identifying and extracting a text area in the financial form from the fused feature maps; A text recognition module, used to extract text sequence features in the text area, and transcribe the extracted text sequence features based on an attention mechanism to extract the text sequence in the text area; The financial nested named entity recognition module is used to encode and semantically enhance the text sequence to obtain a comprehensive vector sequence, and decode the comprehensive vector sequence to identify the named entities in the financial form.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the financial form recognition method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the financial form recognition method according to any one of claims 1 to 8.

Images