Financial information processing method, device, equipment, storage medium and product
Through the integration of financial image features and text features by the attention mechanism model, the problem of low accuracy of financial feature extraction is solved, more efficient financial information recognition is achieved, and the accuracy and efficiency of financial information extraction is improved.
Patent Information
- Application Number
- CN202210242946.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-11
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-03-11
AI Technical Summary
The accuracy of financial feature extraction is not high. The existing technology usually uses natural language processing or computer vision technology alone to identify text files and image files, resulting in excessive feature dimensions, huge calculation amounts, and low recognition accuracy and speed.
The attention mechanism model is used to integrate financial image features and text features, reduce feature dimensions, improve feature accuracy, and obtain a more comprehensive feature association information by integrating the financial text features of text files with the financial image features of image files. Target financial characteristics, thereby achieving accurate identification of the documents to be analyzed.
It improves the accuracy and efficiency of financial information extraction, obtains more accurate target financial information, has higher recognition accuracy, and has more comprehensive feature fusion.
Smart Images

Figure CN114626455B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of deep learning in the field of artificial intelligence, and in particular to a financial information processing method, device, equipment, storage medium and product. Background Art
[0002] In the financial field, more and more online processing content is involved in bank risk control, pre- and post-loan audits, etc. In online processing in the financial field, the user's financial characteristics can be used to identify the user, such as conducting online audits of the user, financial Q&A and other entity processing. Financial characteristics can be related content that characterizes the user's financial status. Usually, financial characteristics can be extracted from documents such as credit reports, income certificates, bank statements, loan information, etc. Therefore, how to accurately obtain financial characteristics is a technical problem that needs to be solved urgently. Summary of the invention
[0003] The present disclosure provides a financial information processing method, device, equipment, storage medium and product for use in the financial field.
[0004] According to a first aspect of the present disclosure, there is provided a financial information processing method, comprising:
[0005] Obtaining a file to be analyzed corresponding to the first user;
[0006] Divide the files to be analyzed into text files and image files according to the file types;
[0007] Extracting financial text features of the text file and extracting financial image features of the image file;
[0008] According to feature association information between the financial text feature and the financial image feature, feature fusion is performed on the financial text feature and the financial image feature to obtain a target financial feature of the file to be analyzed;
[0009] According to the target financial features of the file to be analyzed, target financial information of the file to be analyzed is identified.
[0010] According to a second aspect of the present disclosure, there is provided a financial information processing device, comprising:
[0011] A file acquisition unit, used to acquire a file to be analyzed corresponding to the first user;
[0012] A type classification unit, used for classifying the files to be analyzed into text files and image files according to the file types;
[0013] A feature extraction unit, used to extract financial text features of the text file and financial image features of the image file;
[0014] a feature association unit, configured to perform feature fusion on the financial text feature and the financial image feature according to feature association information between the financial text feature and the financial image feature, so as to obtain a target financial feature of the file to be analyzed;
[0015] The feature recognition unit is used to recognize the target financial information of the file to be analyzed according to the target financial feature of the file to be analyzed.
[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0017] at least one processor; and
[0018] a memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.
[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.
[0021] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.
[0022] The technology disclosed in the present invention solves the problem of low accuracy in extracting financial features. The files to be analyzed are divided into text files and image files according to the file type to extract the financial text features of the text files and the financial image features of the image files. According to the feature association information between the financial text features and the financial image features, the financial text features and the financial image features can be feature fused to obtain the target financial features of the files to be analyzed. According to the target financial features of the files to be analyzed, the target financial information of the files to be analyzed is identified. By utilizing the feature association information between the financial text features and the financial image features, the target financial features obtained by feature fusion of the financial text features and the financial image features can better integrate the two features. The target financial features contain more comprehensive information. The target financial features are used for information identification, and the identification accuracy is higher. More accurate target financial information is obtained, thereby improving the accuracy of financial information extraction.
[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0025] Figure 1 is a scenario diagram of a financial information processing method provided according to an embodiment of the present disclosure;
[0026] Figure 2 is a flow chart of an embodiment of a financial information processing method provided according to an embodiment of the present disclosure;
[0027] Figure 3 is a flowchart of another embodiment of a financial information processing method provided according to an embodiment of the present disclosure;
[0028] Figure 4 is a feature calculation example diagram provided according to an embodiment of the present disclosure;
[0029] Figure 5 is another example diagram of feature calculation provided according to an embodiment of the present disclosure;
[0030] Figure 6 is a flowchart of another embodiment of a financial information processing method provided according to an embodiment of the present disclosure;
[0031] Figure 7 is a structural schematic diagram of an embodiment of a financial information processing device provided according to an embodiment of the present disclosure;
[0032] Figure 8 The present invention is a block diagram of an electronic device used to implement the financial information processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0034] The technical solution of the disclosed embodiment can be applied to the scenario of extracting financial information. By classifying text and images and fusing features according to feature association relationships, accurate target financial features of the file to be analyzed are obtained, and then the accurate target financial features are used to accurately identify the target financial information of the file to be analyzed.
[0035] In the related technology, when identifying financial information for different types of files such as text files and image files, natural language processing (full name: Natural Language Processing, referred to as: NLP) technology is usually used to identify financial text features of text files, and CV (Computer Vision) technology is used to identify financial image features of image files. These two methods are usually used alone, that is, financial text features or financial image features are obtained separately. In some scenarios, financial text features and financial image features can be directly spliced, resulting in too high feature dimensions and huge calculations, resulting in low recognition accuracy and speed.
[0036] In order to solve the above technical problems, the present disclosure considers using an attention mechanism model to fuse financial image features and file features, reduce feature dimensions, improve feature accuracy, and thereby promote the recognition of target financial information.
[0037] The present disclosure provides a financial information processing method, device, equipment, storage medium and product, which are applied to the deep learning field in the field of artificial intelligence to achieve accurate extraction of financial features, and then accurately analyze the financial features, so as to achieve the purpose of improving the analysis accuracy of financial information.
[0038] In the disclosed embodiment, when the files to be analyzed of the first user are obtained, the files to be analyzed can be divided into text files and image files according to the file type, so as to extract the financial text features of the text files and the financial image features of the image files. According to the feature association information between the financial text features and the financial image features, the financial text features and the financial image features can be feature fused to obtain the target financial features of the files to be analyzed. According to the target financial features of the files to be analyzed, the target financial information of the files to be analyzed is identified. By utilizing the feature association information between the financial text features and the financial image features, the target financial features obtained by feature fusion of the financial text features and the financial image features can better integrate the two features, the target financial features contain more comprehensive information, and the target financial features are used for information identification, so that the identification accuracy is higher and more accurate target financial information is obtained.
[0039] The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings.
[0040] For ease of understanding, Figure 1 A scenario diagram of a financial information processing method provided by an embodiment of the present disclosure. In the application scenario of the financial information processing method, a first user device 1, a server 2, and a second user device 3 may be included. The first user device 1 may establish a communication connection with the server 2 via a wired or wireless connection. The second user device 3 may establish a communication connection with the server 2 via a wired or wireless connection.
[0041] Among them, the first user device 1 can face the first user, collect the files to be analyzed of the first user device, and send the files to be analyzed to the server 2. The server 2 can receive the files to be analyzed sent by the first user device 1. The financial image features of the image files in the files to be analyzed are extracted, and the financial text features of the text files are extracted. The feature association information between the financial image features and the financial text features can be used to more accurately fuse the two features to obtain the target financial features. The target financial features can contain more comprehensive features, which can realize the accurate identification of the target financial information and obtain accurate target financial information. After the server 2 identifies and obtains the target financial information, it can send the target financial information to the second user device 3, and the second user device 3 can output the target financial information.
[0042] like Figure 2 As shown, it is a flowchart of an embodiment of a financial information processing method provided by an embodiment of the present disclosure. The method can be configured as a financial information processing device, and the financial information processing device can be located in an electronic device. The financial information processing method can include the following steps:
[0043] 201: Obtain a file to be analyzed corresponding to the first user.
[0044] The file to be analyzed corresponding to the first user can be collected from the bank account information of the first user. The file to be analyzed can be a financial material file, including a credit report, an income certificate, a bank statement, other materials, etc. The file to be analyzed can be account information related to the financial status of the first user.
[0045] The file to be analyzed may be uploaded to the electronic device by the first user. It may also be collected by the electronic device from a third-party financial system based on the account information of the first user. The third-party financial system may include various banking systems, credit reporting systems, etc.
[0046] 202: Classify the files to be analyzed into text files and image files according to the file types.
[0047] The files to be analyzed may include multiple files. The multiple files to be analyzed may be divided into at least one text file and at least one image file according to the file type. The at least one text file may be used as an extraction object for financial text features, and the financial text features of the user are extracted from the at least one text file. Similarly, the at least one image file may be used as an extraction object for financial image features, and the financial image features of the user are extracted from the at least one image file.
[0048] 203: Extracting financial text features from the text file and extracting financial image features from the image file.
[0049] Optionally, extracting the financial text features of the text file may include: extracting all the text in the text file, extracting sentences containing target keywords from all the text, and performing word vector conversion on the extracted sentences to obtain the financial text features.
[0050] Optionally, extracting the financial image features of the image file may include: performing feature calculation on the financial image, for example, performing convolution calculation on the financial image using an image convolution kernel to obtain the financial image features.
[0051] 204: Based on the feature association information between the financial text feature and the financial image feature, the financial text feature and the financial image feature are feature-fused to obtain the target financial feature of the file to be analyzed.
[0052] The feature association relationship can be the attention level or similarity level between the financial text feature and the financial image feature. The higher the feature association, the higher the attention level between the financial text feature and the financial image feature. The feature association relationship can be used to fuse the financial text feature and the financial image feature, improve the calculation between the attention levels of the financial text feature and the financial image feature, and realize the association fusion of the financial text feature and the financial image feature. Compared with directly fusing the financial text feature with the financial image feature, the fusion content is more comprehensive and the target financial feature obtained is more accurate.
[0053] 205: Identify target financial information of the file to be analyzed according to the target financial features of the file to be analyzed.
[0054] According to the target financial characteristics of the file to be analyzed, identifying the target financial information of the file to be analyzed may include: determining the target content information of the file to be analyzed according to the target financial characteristics of the file to be analyzed, determining the analysis target of the first user, analyzing the analysis target according to the content information, and obtaining the analysis result as the target financial information.
[0055] In the disclosed embodiment, when the files to be analyzed of the first user are obtained, the files to be analyzed can be divided into text files and image files according to the file type, so as to extract the financial text features of the text files and the financial image features of the image files. According to the feature association information between the financial text features and the financial image features, the financial text features and the financial image features can be feature fused to obtain the target financial features of the files to be analyzed. According to the target financial features of the files to be analyzed, the target financial information of the files to be analyzed is identified. By utilizing the feature association information between the financial text features and the financial image features, the target financial features obtained by feature fusion of the financial text features and the financial image features can better integrate the two features, the target financial features contain more comprehensive information, and the target financial features are used for information identification, so that the identification accuracy is higher and more accurate target financial information is obtained.
[0056] As an embodiment, before the financial text features and the financial image features are feature-fused according to the feature association information between the financial text features and the financial image features to obtain the target financial features of the file to be analyzed, the method further includes:
[0057] The financial text features and the financial image features are input into the attention mechanism model to obtain the feature correlation information between the financial text features and the financial image features.
[0058] Optionally, an attention mechanism model (full name in English: Attention Mechanism Model) is a mathematical model that can be used to calculate the attention or similarity between financial text features and financial image features. It can filter out effective information between financial text features and financial image features, filter out irrelevant information between the two, and solve the problem of information overload.
[0059] In the disclosed embodiment, the financial text features and the financial image features can be input into the attention mechanism model to obtain the feature association relationship between the financial text features and the financial image features. The feature association information can be accurately and effectively extracted through the attention mechanism model.
[0060] In a possible design, the feature association information may include a first influence weight and a second influence weight. After the financial text features and the financial image features are input into the attention mechanism model to obtain the feature association information between the financial text features and the financial image features, the feature association information may be used to effectively extract the target financial features.
[0061] like Figure 3 FIG. 1 is a flowchart of another embodiment of a financial information processing method provided by the present disclosure, and the method may include the following steps:
[0062] 301: Obtain the to-be-analyzed file corresponding to the first user.
[0063] Some steps in the embodiments of the present disclosure are the same as some steps in the aforementioned embodiments, and for the sake of brevity of description, they are not repeated here.
[0064] 302: Classify the files to be analyzed into text files and image files according to the file types.
[0065] 303: Extracting financial text features from the text file and extracting financial image features from the image file.
[0066] 304: Input the financial text features and the financial image features into the attention mechanism model to obtain a first influence weight and a second influence weight between the financial text features and the financial image features.
[0067] The first influence weight and the second influence weight can be obtained by calculating the attention or similarity of the financial text features and the financial image features by the attention mechanism model, so as to effectively extract the attention of the financial text features and the financial image features.
[0068] The first influence weight may represent the influence degree of the financial image feature on the financial text feature. The second influence weight may represent the influence degree of the financial text feature on the financial image feature.
[0069] 305: According to the first influence weight, a fusion calculation is performed on the financial text feature and the financial image feature to obtain a first correlation feature.
[0070] The first associated feature may be obtained by performing feature fusion calculation on the financial text feature and the financial image feature using the first influence weight.
[0071] 306: According to the second influence weight, the financial text feature and the financial image feature are fused and calculated to obtain a second correlation feature.
[0072] The second associated feature may be obtained by performing feature fusion calculation on the financial text feature and the financial image feature using the second influence weight.
[0073] 307: Concatenate the first correlation feature and the second correlation feature to obtain a target financial feature of the file to be analyzed.
[0074] 308: Identify target financial information of the file to be analyzed according to the target financial features of the file to be analyzed.
[0075] In the disclosed embodiment, after the financial text features and the financial image features are input into the attention mechanism model, the first influence weight and the second influence weight between the financial text features and the financial image features can be obtained. The financial text features and the financial image features can be fused and calculated according to the first influence weight to obtain the first association feature. The financial text features and the financial image features are fused and calculated according to the second influence weight to obtain the second association feature. The first association feature and the second association feature include both text and image features, and the association relationship between the two features is accurately calculated, so that the target financial features obtained are more accurate. Therefore, the target financial information of the file to be analyzed can be identified according to the target financial features of the file to be analyzed, thereby improving the analysis efficiency of the target financial information.
[0076] In a possible design, the financial text features and the financial image features are input into the attention mechanism model to obtain feature association information between the financial text features and the financial image features, including:
[0077] The financial text features and the financial image features are input into the attention mechanism model to obtain feature association information consisting of the text content sub-features corresponding to the financial text features, the image content sub-features corresponding to the financial image features, the first influence weights and the second influence weights.
[0078] According to the first influence weight, the financial text feature and the financial image feature are fused and calculated to obtain the first correlation feature, including:
[0079] According to the first influence weight, weighted summation is performed on the text content sub-feature and the image content sub-feature to obtain a first correlation feature;
[0080] According to the second influence weight, the financial text feature and the financial image feature are fused and calculated to obtain the second correlation feature, including:
[0081] According to the second influence weight, a weighted sum is performed on the text content sub-feature and the image content sub-feature to obtain a second associated feature.
[0082] The first influence weight W1 may include a first sub-weight w1 and a second sub-weight w2. The second influence weight W2 may include a third sub-weight w1' and a fourth sub-weight w2'. Assume that the image content sub-feature is represented by z1 and the text content sub-feature is represented by z2.
[0083] Optionally, according to the first influence weight, weighted summation of text content sub-features and image content sub-features is performed to obtain a first associated feature, which may include: calculating the matrix product of the first sub-weight w1 and the image content sub-feature z1 to obtain the first sub-feature z1'1; calculating the matrix product of the second sub-weight w2 and the text content sub-feature z2 to obtain the second sub-feature z2'1; adding the first sub-feature z1'1 and the second sub-feature z2'1 to obtain the first associated feature V1.
[0084] Optionally, according to the second influence weight, weighted summation of the text content sub-features and the image content sub-features is performed to obtain the second associated feature, which may include: calculating the matrix product of the third sub-weight w1' and the image content sub-feature z1 to obtain the third sub-feature z1'2; calculating the matrix product of the fourth sub-feature w2' and the text content sub-feature z2 to obtain the fourth sub-feature z2'2; adding the third sub-feature z1'2 and the fourth sub-feature z2'2 to obtain the second associated feature V2.
[0085] In a possible design, before obtaining the first associated feature by weighting and summing the text content sub-features and the image content sub-features according to the first influence weight, the method further includes: performing weight normalization processing on the first sub-weight w1 and the second sub-weight w2 in the first influence weight to obtain the normalized first sub-weight w1 and the second sub-weight w2. The text content sub-feature z2 and the image content sub-feature z1 are weighted and summed using the normalized first sub-weight w1 and the second sub-weight w2 to obtain the first associated feature. The w1 and w2 participating in the sub-feature calculation can be the normalized w1 and w2.
[0086] According to the second influence weight, the text content sub-features and the image content sub-features are weighted and summed, and before obtaining the second associated feature, the method further includes: weight normalizing the third sub-weight w1' and the fourth sub-weight w2' in the second influence weight to obtain the normalized third sub-weight w1' and the fourth sub-weight w2'. The text content sub-features and the image content sub-features are weighted and summed using the normalized third sub-weight w1' and the fourth sub-weight w2' to obtain the second associated feature. The w1' and w2' involved in the sub-feature calculation can be the normalized w1' and w2'.
[0087] In the first associated feature, the first influence weight is used to perform feature fusion calculation on the text content sub-feature and the image content sub-feature, and the obtained first associated feature comprehensively includes both the text and the image content, making the first associated feature corresponding to the text more accurate and more effective. Similarly, in the second associated feature, the second influence weight is used to perform feature fusion calculation on the text content sub-feature and the image content sub-feature, and the obtained second associated feature also comprehensively includes both the text and the image content, making the second associated feature corresponding to the image more accurate and more effective.
[0088] In the disclosed embodiment, when the association relationship between the financial text features and the financial image features is calculated using the attention mechanism, the financial text features and the financial image features can be input into the attention mechanism model to obtain the text content sub-features corresponding to the financial text features, the image content sub-features corresponding to the financial image features, and the feature association information composed of the first influence weight and the second influence weight. By calculating the financial text features and the financial image features through the attention mechanism model, the text content sub-features and the image content sub-features, as well as the feature association relationship can be accurately obtained. Through the first influence weight, the financial text features and the financial image features can be weighted and summed to achieve accurate calculation of the first association feature. Through the second influence weight, the financial text features and the financial image features can be weighted and summed to achieve accurate calculation of the second association feature. Through the first influence weight and the second influence weight, the text content sub-features and the graphic content sub-features can be accurately calculated to achieve rapid and effective extraction of the first association feature and the second association feature.
[0089] In a possible design, the attention mechanism model includes: a query matrix, a queried matrix, and a content matrix;
[0090] The financial text features and the financial image features are input into the attention mechanism model to obtain the text content sub-features corresponding to the financial text features, the image content sub-features corresponding to the financial image features, the first influence weights and the second influence weights, which are the feature association information, including:
[0091] Perform matrix calculations on the financial image feature, the query matrix and the queried matrix respectively to obtain a first query feature and a first queried feature corresponding to the financial image feature;
[0092] Perform matrix calculations on the financial text feature, the query matrix and the queried matrix respectively to obtain a second query feature and a second queried feature corresponding to the financial text feature;
[0093] The first query feature is respectively calculated with the first queried feature and the second queried feature to obtain a first influence weight; the first influence weight is the influence degree of the financial image feature on the financial text feature;
[0094] The second query feature is respectively calculated with the first query feature and the second query feature to obtain a second influence weight; the second influence weight is the influence degree of the financial text feature on the financial image feature;
[0095] Based on the financial image features and content matrix, the image content sub-features are calculated.
[0096] According to the financial text features and content matrix, the text content sub-features are calculated.
[0097] Determine feature association information consisting of the first impact weight and the second impact weight.
[0098] Optionally, the query matrix can represent the standard that the text or image itself expects to search. The queried matrix can represent the standard for the other party to search in the text or image, that is, the standard that the other party expects to search. The content matrix can represent the actual existing content expressed in the text or image. The query matrix can be used to calculate the features that the financial image features or financial text features itself expect to search for. The queried matrix can be used to calculate the content that the other party expects to search for in the financial image features or financial text features.
[0099] For ease of understanding, refer to Figure 4 In the feature calculation example diagram shown, the financial text feature is represented by an image vector 401, and the financial text feature is represented by a text vector 402. The query matrix X in the attention mechanism model is represented by a matrix 403, the queried matrix Y is represented by a matrix 404, and the content matrix Z is represented by a matrix 405.
[0100] The financial image feature, i.e., the image vector 401 and the query matrix X 403 are calculated to obtain the first query feature x1 406; the image vector 401 and the query matrix Y 404 are calculated to obtain the first query feature y1 407. The image vector 401 and the content matrix Z 405 are calculated to obtain the image content sub-feature z1 408.
[0101] The financial text feature, that is, the text vector 402 and 403 query matrix X matrix calculation to obtain the first query feature x2, that is, 409; the text vector 402 and 404 are queried matrix X matrix calculation to obtain the second queried feature y2, that is, 410. The text vector 402 and 405 content matrix Z matrix calculation to obtain the text content sub-feature z2, that is, 411.
[0102] The financial image feature is subjected to matrix calculation with the query matrix to obtain the degree of attention paid by the financial image feature to itself, and the first query feature x1 is obtained. The financial image feature is subjected to matrix calculation with the query matrix to obtain the degree of attention paid by the text to the financial image feature, and the first query feature y1 is obtained.
[0103] By calculating the financial text feature and the query matrix, the attention degree of the financial text feature to itself can be obtained, and the second query feature x2 can be obtained. By performing matrix calculation on the financial text feature and the query matrix, the attention degree of the financial text feature to the image can be obtained, and the second query feature y2 can be obtained.
[0104] The first sub-weight w1 is obtained by performing matrix dot multiplication on the first query feature x1 and the first queried feature y1, and the second sub-weight w2 is obtained by performing matrix dot multiplication on the first query feature x1 and the second queried feature y2. The first influence weight composed of the first sub-weight w1 and the second sub-weight w2 is determined.
[0105] The third sub-weight w1' is obtained by performing matrix dot multiplication on the second query feature x2 and the first query feature y1, and the fourth sub-weight w2' is obtained by performing matrix dot multiplication on the second query feature x2 and the second query feature y2. The second influence weight composed of the third sub-weight w1' and the fourth sub-weight w2' is determined.
[0106] The text content sub-feature z2 may be the matrix product of the financial text feature and the content matrix z. The image content sub-feature z1 may be the matrix product of the financial image feature and the content matrix z.
[0107] For ease of understanding, Figure 4 Based on x1, x2, y1, y2, z1, and z2, the target financial features of the file to be analyzed can be comprehensively extracted.
[0108] refer to Figure 5 In the example diagram of feature calculation shown in the figure, the first influence weight may include: the first sub-weight 501: w1 = x1·y1 and the second sub-weight 502: w2 = x1·y2. The first sub-weight w1 and the second sub-weight w2 in the first influence weight may be weight normalized to obtain the normalized first sub-weight 503 and the normalized second sub-weight 504. The matrix product of the normalized w1 and the image content sub-feature z1 is calculated to obtain the first sub-feature 505, and the formula is z1'1 = w1*z1. The matrix product of the normalized w2 and the text content sub-feature z2 is calculated to obtain the second sub-feature 506, and the formula is z2'1 = w2*z2; the first sub-feature z1'1 and the second sub-feature z2'1 are added to obtain 507 the first associated feature V1.
[0109] The second influence weight may include: a third sub-weight 508: w1'=x2·y1 and a fourth sub-weight 509: w2'=x2·y2. The third sub-weight w1' and the fourth sub-weight w2' in the second influence weight may also be weight normalized to obtain a normalized third sub-weight 510 and a fourth sub-weight 511. The matrix product of the normalized w1' and the image content sub-feature z1 is calculated to obtain a third sub-feature 512, and the formula is z1'2=w1'*z1. The matrix product of the normalized w2' and the text content sub-feature z2 is calculated to obtain a fourth sub-feature 513, and the formula is z2'2=w1'*z2. The third sub-feature z1'2 and the fourth sub-feature z2'2 are added to obtain 514 the second associated feature V2.
[0110] The first correlation feature 507 and the second correlation feature 514 are concatenated to obtain a target financial feature 515 of the file to be analyzed. According to the target financial feature 515 of the file to be analyzed, target financial information 516 of the file to be analyzed is identified.
[0111] In one possible design, the query matrix x, the queried matrix y and the content matrix z of the attention mechanism model can be obtained through training. The training process of the attention mechanism model is the same as the input object, financial image features and financial text features in the embodiment of the present disclosure. In addition, the training process of the entire model can also include an extraction model of financial image features, an extraction model of financial text features, and an identification model of the obtained target financial features. The training data can include training files of multiple training users and label data corresponding to the training files. During the model training process, the input and output of the models such as file processing, feature extraction, and information recognition, and the data processing process are the same as the financial information processing method of the present disclosure.
[0112] In the disclosed embodiment, the financial image feature is subjected to matrix calculation with the query matrix and the queried matrix respectively, and the first query feature and the first queried feature corresponding to the financial image feature can be obtained. The financial text feature is subjected to matrix calculation with the query matrix and the queried matrix respectively, and the second query feature and the second queried feature corresponding to the financial text feature can be obtained. The first query feature and the second query feature are associated with the query content of the feature. The first queried feature and the second queried feature can calculate the degree of attention between the financial image feature and the financial text feature, and the obtained first influence weight and the second influence weight can reflect the degree of attention between the financial image feature and the financial text feature. The obtained first influence weight and the second influence weight can more accurately analyze the degree of association between the financial image feature and the financial text feature, and obtain accurate feature association.
[0113] As an embodiment, the files to be analyzed are divided into text files and image files according to the file type, including:
[0114] Determine the file format of the file to be analyzed;
[0115] According to the file format of the file to be analyzed, the text file corresponding to the text format and the image file corresponding to the image format are determined to complete the division of the text file and the image file.
[0116] Optionally, the file formats of the files to be analyzed may include: pdf (Portable Document Format), word (Microsoft Word), jpg (Joint Photographic Experts Group), png (Portable Network Graphics), etc. Among them, pdf, word, etc. formats are non-image formats and can be determined as text formats. Jpg, png, etc. formats are image formats and can be determined as image formats.
[0117] There may be multiple files to be analyzed, and file formats corresponding to the multiple files to be analyzed may be determined. From the file formats corresponding to the multiple files to be analyzed, a text file corresponding to the text format and an image file corresponding to the image format may be determined.
[0118] In the disclosed embodiment, when dividing the files to be analyzed, the file types can be divided according to the file formats of the files to be analyzed. The file formats can be quickly obtained from the file attributes. By dividing the files according to the file formats, the file division efficiency and the file division accuracy can be quickly determined.
[0119] In order to obtain accurate text features, in a possible design, when obtaining, the text file includes at least one, extracting financial text features of the text file includes:
[0120] Input at least one text file into the corresponding word vector extraction model to obtain the text word vector corresponding to the at least one text file.
[0121] The text word vectors corresponding to at least one text file are merged to obtain financial text features.
[0122] Optionally, the word vector extraction model may include an embedding model. The embodiments of the present disclosure do not impose too many restrictions on the specific type of the word vector extraction model.
[0123] In the disclosed embodiment, the word vector extraction model can be used to accurately extract the word vectors of the text file. By fusing the word vectors corresponding to at least one text file, the obtained financial text features contain more comprehensive word vectors of the text files, and the financial text features are more accurate and have higher feature expression.
[0124] In a possible design, financial image features of an image file are extracted, including:
[0125] Using a bottom-up downsampling method, downsampling the image file at least once based on the convolution kernel to obtain at least one convolution feature;
[0126] Sort at least one convolution feature in ascending order of size;
[0127] Using a top-down upsampling method, starting from the second convolution feature, the convolution features are upsampled to obtain sampling features corresponding to at least one convolution feature.
[0128] The sampling features corresponding to at least one convolution feature are concatenated to obtain image features.
[0129] Using a bottom-up downsampling method, downsampling the image file at least once based on the convolution kernel, obtaining at least one convolution feature may include: convolving the image file with the convolution kernel to obtain the first convolution feature, and for any subsequent downsampling step: continuing to convolve the convolution feature obtained by the previous sampling with the convolution kernel to obtain a new convolution feature, until the sampling number reaches the target convolution number, and obtaining the convolution features obtained by at least one downsampling.
[0130] Using a top-down upsampling method, starting from the second convolution feature, upsampling the convolution feature, and obtaining the sampling features corresponding to at least one convolution feature may include: using a top-down upsampling method, determining the first convolution feature as the sampling feature; starting from the second convolution feature, for the upsampling steps of any convolution feature: determining the current convolution feature and the previous convolution feature of the current convolution kernel; upsampling the previous convolution feature according to the size of the current convolution feature to obtain an intermediate feature; adding the intermediate feature to the current convolution feature to obtain the current sampling feature. Until the upsampling of the last convolution feature is completed, the sampling features corresponding to at least one convolution feature are obtained;
[0131] Optionally, the convolution kernel may be a model parameter of a mapping model or an extraction model of image features.
[0132] The size of the convolution kernel can be 2*2. The convolution kernel of size 2*2 can be convolved with the convolution feature input each time to obtain a new convolution feature. When using a 2*2 convolution kernel for convolution, the size of the newly obtained convolution feature is half the size of the input convolution feature. For example, when the image size is 224*224, after it is convolved with a 2*2 convolution kernel, the size of the new convolution feature obtained is 112*112, and the size of the convolution feature after it is 56*56. Similarly, the sizes of the new convolution features after the 56*56 convolution feature can be: 28*28, 14*14, 7*7 and 3*3 respectively. In one possible design, the convolution kernel can include multiple convolution kernels, and the size of the convolution kernel can remain unchanged, which can be 2*2. Of course, the size of the convolution kernel can be set according to actual usage requirements. The 2*2 in the embodiment of the present disclosure is only schematic and should not constitute a limitation to the technical solution of the present disclosure.
[0133] For ease of understanding, when upsampling, assuming that the current convolution feature size is 7*7, and the size of the previous convolution feature of the convolution feature is 3*3, the previous convolution feature 3*3 can be upsampled to a size of 7*7 to obtain a 7*7 intermediate feature, and the 7*7 intermediate feature is added to the current 7*7 convolution feature to obtain a 7*7 sampling feature. Adding the intermediate feature to the current convolution feature specifically refers to adding the feature values of the same coordinate to obtain the feature value of the coordinate in the sampling feature, and the feature values of all coordinates are added to obtain the sampling feature.
[0134] In a possible design, performing feature splicing on sampling features corresponding to at least one convolution feature to obtain image features may include: converting the convolution features into convolution vectors, obtaining convolution vectors corresponding to at least one convolution feature, and performing vector splicing on convolution vectors corresponding to at least one convolution feature to obtain image features. The image features include sampling features of different sizes, have higher feature coverage, and the obtained image features are more comprehensive and more accurate.
[0135] In the disclosed embodiment, when extracting financial image features from an image file, convolution kernels of multiple scales may be used, and a bottom-up downsampling method may be used for convolution calculation, and then a top-down method may be used for upsampling. During the sampling process, features of different sizes can be collected more efficiently. After obtaining the sampling features corresponding to the multiple convolution kernels, the sampling features corresponding to the multiple convolution kernels may be spliced to obtain financial image features. The financial image features include features of different sizes, and the obtained financial image features are more comprehensive and accurate.
[0136] like Figure 6FIG. 1 is a flowchart of another embodiment of a financial information processing method provided by the present disclosure, and the method may include the following steps:
[0137] 601: Obtain a file to be analyzed corresponding to a first user.
[0138] It should be noted that some steps in the embodiments of the present disclosure are the same as some steps in the aforementioned embodiments, and will not be repeated here for the sake of brevity of description.
[0139] 602: Classify the files to be analyzed into text files and image files according to the file types;
[0140] Extract financial text features from text files and extract financial image features from image files.
[0141] 603: According to the feature association information between the financial text feature and the financial image feature, the financial text feature and the financial image feature are feature-fused to obtain the target financial feature of the file to be analyzed.
[0142] 604: Input the target financial features corresponding to the file to be analyzed into the language recognition model to obtain target content information corresponding to the target financial features.
[0143] 605: Analyze and process the target content information according to the financial analysis request provided by the second user to obtain target financial information.
[0144] The second user may send a financial analysis request to the electronic device through the second user device. The financial analysis request may include the user information of the first user and the analysis content of the first user. The analysis content may include, for example, loan approval, financial questions and answers, qualification review, etc. The target content information may include, for example, that the first user has never been overdue, the asset status of the first user, the loan information of the first user, etc. The target financial information may be the analysis result obtained by analyzing the financial analysis request using the target content information.
[0145] 606: Output the target financial information to the second user.
[0146] Outputting the target financial information to the second user may include: sending the target financial information to a second user device of the second user. The second user device may output the received target financial information.
[0147] In the disclosed embodiment, the target financial feature of the file to be analyzed is obtained, and the target financial feature can be input into the language recognition model to obtain the target content information corresponding to the target financial feature. According to the financial analysis request provided by the second user, the target content information can be analyzed and processed to obtain the target financial information, which is the accurate analysis result of the target content information obtained on the basis of the target financial feature corresponding to the target content information. By outputting the target financial information to the second user, the accurate use of the second user can be achieved, and the use efficiency of the second user can be improved.
[0148] like Figure 7 FIG. 1 is a schematic diagram of a structure of an embodiment of a financial information processing device provided by an embodiment of the present disclosure. The financial information processing device 700 may include the following units:
[0149] File acquisition unit 701: used to acquire a file to be analyzed corresponding to a first user;
[0150] Type classification unit 702: used to classify the files to be analyzed into text files and image files according to the file types;
[0151] Feature extraction unit 703: used to extract financial text features of text files and financial image features of image files;
[0152] Feature association unit 704: used to perform feature fusion on the financial text feature and the financial image feature according to feature association information between the financial text feature and the financial image feature, so as to obtain target financial features of the file to be analyzed;
[0153] Feature recognition unit 705: used to recognize target financial information of the file to be analyzed according to the target financial feature of the file to be analyzed.
[0154] In the disclosed embodiment, when the files to be analyzed of the first user are obtained, the files to be analyzed can be divided into text files and image files according to the file type, so as to extract the financial text features of the text files and the financial image features of the image files. According to the feature association information between the financial text features and the financial image features, the financial text features and the financial image features can be feature fused to obtain the target financial features of the files to be analyzed. According to the target financial features of the files to be analyzed, the target financial information of the files to be analyzed is identified. By utilizing the feature association information between the financial text features and the financial image features, the target financial features obtained by feature fusion of the financial text features and the financial image features can better integrate the two features, the target financial features contain more comprehensive information, and the target financial features are used for information identification, so that the identification accuracy is higher and more accurate target financial information is obtained.
[0155] As an embodiment, it also includes:
[0156] The association calculation unit is used to input the financial text features and the financial image features into the attention mechanism model to obtain feature association information between the financial text features and the financial image features.
[0157] In a possible design, the feature association information includes: a first influence weight and a second influence weight; and the feature association unit includes:
[0158] A first calculation module, configured to perform a fusion calculation on the financial text feature and the financial image feature according to the first influence weight to obtain a first correlation feature;
[0159] A second calculation module, configured to perform a fusion calculation on the financial text feature and the financial image feature according to the second influence weight to obtain a second correlation feature;
[0160] The feature splicing module is used to perform feature splicing on the first associated feature and the second associated feature to obtain the target financial feature of the file to be analyzed.
[0161] In some embodiments, the association calculation unit includes:
[0162] A feature input module, used to input financial text features and financial image features into an attention mechanism model to obtain feature association information consisting of text content sub-features corresponding to the financial text features, image content sub-features corresponding to the financial image features, a first influence weight, and a second influence weight;
[0163] The first computing module includes:
[0164] A first calculation submodule, configured to weight the text content sub-feature and the image content sub-feature according to a first influence weight to obtain a first correlation feature;
[0165] The second computing module includes:
[0166] The second calculation submodule is used to weight the text content sub-feature and the image content sub-feature according to the second influence weight to obtain a second associated feature.
[0167] As a possible implementation method, the attention mechanism model includes: query matrix, queried matrix and content matrix;
[0168] Feature input module, including:
[0169] A first calculation submodule, used to perform matrix calculations on the financial image feature, the query matrix and the queried matrix respectively, to obtain a first query feature and a first queried feature corresponding to the financial image feature;
[0170] A second calculation submodule is used to perform matrix calculations on the financial text feature, the query matrix and the queried matrix respectively, to obtain a second query feature and a second queried feature corresponding to the financial text feature;
[0171] The third calculation submodule is used to perform feature calculation on the first query feature, the first queried feature and the second queried feature respectively to obtain a first influence weight; the first influence weight is the influence degree of the financial image feature on the financial text feature;
[0172] The fourth calculation submodule is used to perform feature calculation on the second query feature, the first query feature and the second query feature respectively to obtain a second influence weight; the second influence weight is the influence degree of the financial text feature on the financial image feature;
[0173] The first feature submodule is used to calculate and obtain image content subfeatures based on the financial image features and the content matrix;
[0174] The second feature submodule is used to calculate and obtain text content sub-features based on the financial text features and the content matrix;
[0175] The association determination submodule is used to determine feature association information composed of the first influence weight and the second influence weight.
[0176] As yet another embodiment, the type classification unit includes:
[0177] A format determination module, used to determine the file format of the file to be analyzed;
[0178] The format division module is used to determine the text file corresponding to the text format and the image file corresponding to the image format according to the file format of the file to be analyzed.
[0179] In one possible design, the text format includes at least one feature extraction unit, including:
[0180] The first extraction module is used to input at least one text file into a corresponding word vector extraction model to obtain a text word vector corresponding to at least one text file.
[0181] The second extraction module is used to fuse the text word vectors corresponding to at least one text file to obtain financial text features.
[0182] In some embodiments, the feature extraction unit comprises:
[0183] A first convolution module, configured to downsample the image file at least once based on a convolution kernel in a bottom-up downsampling manner to obtain at least one convolution feature;
[0184] A feature sorting module, used to sort at least one convolution feature in ascending order of size to obtain at least one sorted convolution feature;
[0185] A second convolution module is used to upsample the sorted convolution features starting from the second convolution feature in a top-down upsampling manner to obtain sampling features corresponding to at least one convolution feature;
[0186] The feature splicing module is used to perform feature splicing on the sampling features corresponding to at least one convolution feature to obtain image features.
[0187] In some embodiments, the feature recognition unit includes:
[0188] A content recognition module is used to input the target financial features corresponding to the file to be analyzed into the language recognition model to obtain the target content information corresponding to the target financial features;
[0189] A financial analysis module, configured to analyze and process the target content information according to the financial analysis request provided by the second user, and obtain target financial information;
[0190] The device also includes:
[0191] The information output unit is used to output target financial information to the second user.
[0192] The financial information processing device provided in the embodiments of the present disclosure can execute the financial information processing method in the above embodiments. For the specific contents executed by each unit, module, and sub-module, please refer to the description of the embodiment of the method in the above embodiments, which will not be repeated here.
[0193] It should be noted that the file to be analyzed in this embodiment is not a file to be analyzed for a specific user and cannot reflect the personal information of a specific user. It should be noted that the file to be analyzed in this embodiment comes from a public data set.
[0194] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0195] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0196] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.
[0197] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0198] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0199] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0200] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as financial information processing methods. For example, in some embodiments, the financial information processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the financial information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the financial information processing method in any other appropriate manner (e.g., by means of firmware).
[0201] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0202] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0203] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0204] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0205] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0206] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0207] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0208] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A financial information processing method, comprising: Obtaining a file to be analyzed corresponding to the first user; Divide the files to be analyzed into text files and image files according to the file types; Extracting financial text features of the text file and extracting financial image features of the image file; According to feature association information between the financial text feature and the financial image feature, feature fusion is performed on the financial text feature and the financial image feature to obtain a target financial feature of the file to be analyzed; identifying target financial information of the file to be analyzed according to the target financial features of the file to be analyzed; Before the step of fusing the financial text features with the financial image features according to the feature association information between the financial text features and the financial image features to obtain the target financial features of the file to be analyzed, the method further includes: Inputting the financial text features and the financial image features into an attention mechanism model to obtain feature association information between the financial text features and the financial image features; The feature association information includes: a first influence weight and a second influence weight; The step of fusing the financial text feature with the financial image feature according to the feature association information between the financial text feature and the financial image feature to obtain the target financial feature of the file to be analyzed includes: According to the first influence weight, the financial text feature and the financial image feature are fused and calculated to obtain a first correlation feature; According to the second influence weight, the financial text feature and the financial image feature are fused and calculated to obtain a second correlation feature; Performing feature splicing on the first associated feature and the second associated feature to obtain the target financial feature of the file to be analyzed; The step of inputting the financial text features and the financial image features into the attention mechanism model to obtain feature association information between the financial text features and the financial image features includes: Inputting the financial text feature and the financial image feature into the attention mechanism model to obtain feature association information consisting of a text content sub-feature corresponding to the financial text feature, an image content sub-feature corresponding to the financial image feature, the first influence weight, and the second influence weight; The step of fusing and calculating the financial text feature and the financial image feature according to the first influence weight to obtain a first correlation feature includes: performing weighted summation of the text content sub-feature and the image content sub-feature according to the first influence weight to obtain the first association feature; The step of fusing and calculating the financial text feature and the financial image feature according to the second influence weight to obtain a second correlation feature includes: According to the second influence weight, a weighted sum is performed on the text content sub-feature and the image content sub-feature to obtain the second associated feature.
2. The method according to claim 1, wherein: The attention mechanism model includes: a query matrix, a queried matrix and a content matrix; The step of inputting the financial text feature and the financial image feature into the attention mechanism model to obtain feature association information consisting of a text content sub-feature corresponding to the financial text feature, an image content sub-feature corresponding to the financial image feature, the first influence weight, and the second influence weight includes: Performing matrix calculation on the financial image feature, the query matrix and the queried matrix respectively, to obtain a first query feature and a first queried feature corresponding to the financial image feature; Perform matrix calculation on the financial text feature, the query matrix and the queried matrix respectively, to obtain a second query feature and a second queried feature corresponding to the financial text feature; Perform feature calculation on the first query feature, the first queried feature and the second queried feature respectively to obtain the first influence weight; the first influence weight is the influence degree of the financial image feature on the financial text feature; The second query feature is respectively calculated with the first queried feature and the second queried feature to obtain the second influence weight; the second influence weight is the influence degree of the financial text feature on the financial image feature; Calculating and obtaining the image content sub-features according to the financial image features and the content matrix; Calculating and obtaining the text content sub-features according to the financial text features and the content matrix; Determine feature association information consisting of the first impact weight and the second impact weight.
3. The method according to claim 1 or 2, wherein: The step of dividing the files to be analyzed into text files and image files according to the file types includes: Determine the file format of the file to be analyzed; According to the file format of the file to be analyzed, the text file corresponding to the text format and the image file corresponding to the image format are determined to complete the division of the text file and the image file.
4. The method according to claim 3, wherein: The text format includes at least one, and the extracting of financial text features of the text file includes: Inputting at least one text file into a corresponding word vector extraction model to obtain a text word vector corresponding to at least one of the text files; The text word vectors corresponding to at least one text file are merged to obtain the financial text feature.
5. The method according to claim 1 or 2, wherein: The step of extracting the financial image features of the image file comprises: Determine a plurality of convolution kernels; arrange the plurality of convolution kernels in ascending order according to their respective sizes; According to the order of the multiple convolution kernels from small to large, starting from the convolution kernel with the largest size, using a bottom-up downsampling method, the image file is used as input and the convolution kernel with the largest size is convolved to obtain convolution features, and the obtained convolution features are used as input of the next convolution kernel until the convolution of all convolution kernels is completed, and the convolution features corresponding to at least one convolution kernel are obtained; According to the order of the multiple convolution kernels from small to large, starting from the convolution kernel with the smallest size, a current convolution kernel and a previous convolution kernel of the current convolution kernel are determined by a top-down upsampling method; the sampling features of the previous convolution kernel are upsampled according to the size of the current convolution kernel to obtain intermediate features; the intermediate features are added to the convolution features corresponding to the current convolution kernel to obtain the sampling features of the current convolution kernel, until the last convolution kernel sampling is completed, and the sampling features corresponding to the multiple convolution kernels are obtained; The sampling features corresponding to multiple convolution kernels are spliced to obtain the financial image features.
6. The method according to claim 1 or 2, wherein: The step of identifying target financial information of the file to be analyzed according to the target financial feature of the file to be analyzed includes: Inputting the target financial features corresponding to the file to be analyzed into a language recognition model to obtain target content information corresponding to the target financial features; Analyze and process the target content information according to the financial analysis request provided by the second user to obtain the target financial information; The method further comprises: The target financial information is output for the second user.
7. A financial information processing device, comprising: A file acquisition unit, used to acquire a file to be analyzed corresponding to the first user; A type classification unit, used for classifying the files to be analyzed into text files and image files according to the file types; A feature extraction unit, used to extract financial text features of the text file and financial image features of the image file; a feature association unit, configured to perform feature fusion on the financial text feature and the financial image feature according to feature association information between the financial text feature and the financial image feature, so as to obtain a target financial feature of the file to be analyzed; A feature recognition unit, configured to recognize target financial information of the file to be analyzed according to the target financial feature of the file to be analyzed; Also includes: an association calculation unit, configured to input the financial text feature and the financial image feature into an attention mechanism model to obtain feature association information between the financial text feature and the financial image feature; The feature association information includes: a first influence weight and a second influence weight; and the feature association unit includes: A first calculation module, configured to perform a fusion calculation on the financial text feature and the financial image feature according to the first influence weight to obtain a first correlation feature; A second calculation module, configured to perform a fusion calculation on the financial text feature and the financial image feature according to the second influence weight to obtain a second correlation feature; A feature splicing module, used for performing feature splicing on the first associated feature and the second associated feature to obtain a target financial feature of the file to be analyzed; The association calculation unit comprises: A feature input module, used to input the financial text feature and the financial image feature into the attention mechanism model to obtain feature association information consisting of a text content sub-feature corresponding to the financial text feature, an image content sub-feature corresponding to the financial image feature, the first influence weight, and the second influence weight; The first calculation module includes: A first calculation submodule, configured to weight the text content sub-feature and the image content sub-feature according to the first influence weight to obtain the first association feature; The second calculation module includes: The second calculation submodule is used to weight the text content sub-feature and the image content sub-feature according to the second influence weight to obtain the second association feature.
8. The device according to claim 7, wherein: The attention mechanism model includes: a query matrix, a queried matrix and a content matrix; The feature input module comprises: A first calculation submodule, configured to perform matrix calculations on the financial image feature, the query matrix and the queried matrix respectively, to obtain a first query feature and a first queried feature corresponding to the financial image feature; A second calculation submodule, configured to perform matrix calculation on the financial text feature, the query matrix and the queried matrix respectively, to obtain a second query feature and a second queried feature corresponding to the financial text feature; A third calculation submodule is used to perform feature calculation on the first query feature, the first queried feature and the second queried feature respectively to obtain the first influence weight; the first influence weight is the influence degree of the financial image feature on the financial text feature; a fourth calculation submodule, configured to perform feature calculation on the second query feature, the first queried feature and the second queried feature respectively, to obtain the second influence weight; the second influence weight is the influence degree of the financial text feature on the financial image feature; A first feature submodule, configured to calculate and obtain the image content subfeature according to the financial image feature and the content matrix; A second feature submodule, configured to calculate and obtain the text content sub-feature according to the financial text feature and the content matrix; The association determination submodule is used to determine feature association information composed of the first influence weight and the second influence weight.
9. The device according to claim 7 or 8, wherein: The type classification unit comprises: A format determination module, used to determine the file format of the file to be analyzed; The format classification module is used to determine the text file corresponding to the text format and the image file corresponding to the image format according to the file format of the file to be analyzed.
10. The device according to claim 9, wherein: The text format includes at least one, and the feature extraction unit includes: A first extraction module, used to input at least one text file into a corresponding word vector extraction model to obtain a text word vector corresponding to at least one of the text files; The second extraction module is used to fuse the text word vectors corresponding to at least one text file to obtain the financial text features.
11. The device according to claim 7 or 8, wherein: The feature extraction unit comprises: A first convolution module, configured to downsample the image file at least once based on a convolution kernel in a bottom-up downsampling manner to obtain at least one convolution feature; A feature sorting module, used to sort at least one of the convolution features in ascending order of size to obtain at least one sorted convolution feature; A second convolution module is used to upsample the sorted convolution features starting from the second convolution feature in a top-down upsampling manner to obtain a sampling feature corresponding to at least one of the convolution features; The feature stitching module is used to perform feature stitching on the sampling features corresponding to at least one of the convolution features to obtain the image features.
12. The device according to claim 7 or 8, wherein: The feature recognition unit comprises: A content recognition module, used to input the target financial features corresponding to the file to be analyzed into a language recognition model to obtain target content information corresponding to the target financial features; A financial analysis module, configured to analyze and process the target content information according to a financial analysis request provided by a second user, and obtain the target financial information; The device also includes: An information output unit is used to output the target financial information to the second user.
13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.
15. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 6 when executed by a processor.
