Foreign language bill image translation method and device based on artificial intelligence and medium

Abstract

The invention provides a foreign language bill image translation method and device based on artificial intelligence and a medium, and belongs to the technical field of data information. The foreign language bill image translation method comprises the following steps: acquiring a foreign language bill image, inputting the foreign language bill image into an image recognition model, and outputting to obtain bill character information and position coordinate information; inputting the bill character information into a pre-trained bill translation special model, and translating to obtain Chinese bill information; inputting the information into a large language model, and optimizing the Chinese bill information according to a preset bill question and answer prompt word; the position coordinate information is used, the optimized Chinese bill information is arranged and combined according to a Chinese bill format, and a Chinese bill image is drawn; extracting bill key information from the Chinese bill information by using a key information extraction model; and displaying the Chinese bill image and the bill key information. The problem that effective bill information is difficult to train and accurately extract due to the fact that a bill data set is difficult to accurately translate in the prior art can be solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data information, and particularly relates to a foreign language bill image translation method, device and medium based on artificial intelligence. BACKGROUND

[0002] As an important part of modern enterprise data management, financial software not only provides efficient and standardized financial account management for enterprises, but also comprehensively monitors enterprise fund accounts and income and expenditure. To improve the practicality and user experience of financial software, existing financial software begins to integrate bill automatic recognition and input functions. This function automatically recognizes and inputs financial bills, saving the tedious steps of traditional manual input and greatly improving the intuitiveness of financial information. Through this intelligent way, financial software can provide more accurate and efficient financial information support, thereby promoting the development of enterprise financial management in a more intelligent and automatic direction.

[0003] Under the background of economic globalization, many enterprises inevitably encounter foreign language bills when conducting overseas business. With the continuous expansion of overseas markets by enterprises, the existing financial software has exposed some significant defects in the face of a large number of foreign language bills: first, foreign language bills have various formats, and key information can be presented in multiple forms, increasing the complexity of information extraction; when trying to extract key information from bills, existing natural language processing often makes errors, such as incorrect amount recognition, failed date format conversion, or incomplete item description extraction. In addition, there are a large number of professional domain vocabularies in foreign language bills, which often have specific industry meanings, or even polysemy, which further affects translation accuracy. Existing translation models mostly only translate text images, for example: the existing patent CN118552965A provides a text image translation model training method, which encodes and decodes the features of text images and source text by obtaining a pre-training model and training data to generate a translation result; at the same time, calculate the text image translation loss and multi-level knowledge transfer loss, and fuse the two as training loss to update the pre-training model parameters.

[0004] However, foreign language bills often have strong professionalism, such as Figure 1 not only contains polysemous words but also involves the analysis of many similar information points. Since existing translation models are general models and mostly only translate text images, they cannot specifically translate bill data, resulting in poor translation accuracy. As shown in Figure 2 , the existing M2M-100 translation model directly translates Figure 1 as shown in the bill, which will Figure 1The "tip" in the "tip" directly translates to "prompt", rather than the "tip" commonly used in Chinese tickets. The above problems make it difficult for the prior art to quickly and accurately extract effective ticket information, which affects the smoothness and accuracy of financial work. SUMMARY

[0005] The present application aims to provide an artificial intelligence-based foreign language ticket image translation scheme, which can efficiently and accurately translate foreign language ticket images, accurately extract effective ticket information, and improve the processing efficiency, smoothness and accuracy of financial software when processing foreign language tickets.

[0006] According to a first aspect of the present application, the present application provides an artificial intelligence-based foreign language ticket image translation method, comprising: obtaining a foreign language ticket image, inputting the foreign language ticket image into an image recognition model, and outputting ticket text information and position coordinate information; inputting the ticket text information into a pre-trained ticket translation special model to translate Chinese ticket information; inputting the Chinese ticket information into a large language model, and optimizing the language fluency of the Chinese ticket information according to a preset ticket question and answer prompt word; using the position coordinate information, arranging and combining the optimized Chinese ticket information according to the Chinese ticket format, and drawing a Chinese ticket image in Chinese ticket format; using a key information extraction model to extract ticket key information from the Chinese ticket information, wherein the key information extraction model is pre-trained by key word labels; displaying the Chinese ticket image and the ticket key information.

[0007] Preferably, in the above foreign language ticket image translation method, the step of inputting the foreign language ticket image into the image recognition model to output the ticket text information and the position coordinate information comprises: obtaining a recognition model training set, classifying the foreign language ticket images in the recognition model training set according to the language type and format type of the foreign language ticket images; data labeling the classified foreign language ticket images according to the distribution position of the key information in the foreign language ticket images and the type of the key information, and using a clustering algorithm to generate a clustering anchor box corresponding to the key information in the foreign language ticket images; using the data labeled foreign language ticket images containing the clustering anchor box to train the image recognition model in combination with the attention mechanism; when the image recognition model is trained, inputting the obtained foreign language ticket image into the image recognition model; using the image recognition model to extract the ticket text information, the position coordinate information and the confidence from the foreign language ticket image.

[0008] Preferably, in the above foreign bill image translation method, the step of inputting the bill text information into the pre-trained bill translation special model to obtain Chinese bill information includes: According to the language type and format type of the foreign bill image, combined with the position coordinate information, the data type of the bill text information is determined; Classify the bill text information to obtain a translation model training set containing bill text information of multiple data types; The translation model training set is input into the preset translation model, and the preset translation model is trained combined with the preset Chinese bill corpus to obtain a bill translation special model; wherein the preset Chinese bill corpus contains professional Chinese bill corpus corresponding to the data type; The bill text information is input into the bill translation special model, and the Chinese bill information corresponding to the bill text information and the Bool value are output.

[0009] Preferably, in the above foreign bill image translation method, the step of training the preset translation model combined with the preset Chinese bill corpus to obtain the bill translation special model includes: Use the Chinese bill corpus to construct a special semantic space, and use the Chinese bill corpus to train the preset translation model, so that the preset translation model learns the bill translation rules corresponding to the Chinese bill corpus; Control the preset translation model to map the bill text information into professional word vectors in the special semantic space according to the bill translation rules; Combine all professional word vectors corresponding to the bill text information, convert all professional word vectors into Chinese text, and obtain Chinese bill information.

[0010] Preferably, in the above foreign bill image translation method, the step of inputting the Chinese bill information into the large language model and optimizing the language fluency of the Chinese bill information according to the preset bill question and answer prompt words includes: According to the Chinese expression habit, write multiple bill question and answer prompt words, and use the multiple bill question and answer prompt words to build a bill prompt word template; The Chinese bill information is input into the large language model, and the output of the large language model is optimized using the question and answer prompt words in the bill prompt word template to obtain Chinese bill information with optimized language fluency.

[0011] Preferably, in the above foreign bill image translation method, the step of using the position coordinate information to arrange and combine the Chinese bill information optimized according to the Chinese bill format to draw the Chinese bill image includes: Generate a Chinese bill framework according to the Chinese bill format, wherein the Chinese bill framework includes the data type corresponding to the Chinese bill information and the Chinese filling area; According to the correspondence between the position coordinate information and the Chinese filling area, the Chinese bill information is mapped to the data type in the Chinese bill framework.

[0012] Preferably, in the foreign bill image translation method, the step of extracting bill key information from the Chinese bill information using the key information extraction model comprises: The bill key information in the Chinese bill information is labeled using a label labeling tool to obtain a training set; The key information extraction model is trained using the training set until the loss function of the key information extraction model converges; The optimized Chinese bill information is input into the key information extraction model; The key information extraction module is used to extract bill key information from the Chinese bill information.

[0013] Preferably, in the foreign bill image translation method, the step of extracting bill key information from the Chinese bill information using the key information extraction model comprises: The Chinese bill sample is labeled for key information using a Doccano labeling tool to obtain a keyword label; The initial learning rate and the loss function of the key information extraction model are set, the Chinese bill sample containing the keyword label is input into the key information extraction model for multiple rounds of training until the loss function converges; The Chinese bill information is input into the key information extraction model to extract bill key information.

[0014] According to the second aspect of the present application, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the artificial intelligence-based foreign bill image translation method according to any one of the above technical solutions.

[0015] According to the third aspect of the present application, the present application also provides a computer storage medium having a computer program stored thereon, wherein the computer program is executed to implement the artificial intelligence-based foreign bill image translation method according to any one of the above technical solutions.

[0016] The technical solution of the present application has at least the following technical effects: The application provides an AI-based foreign language bill image translation scheme, which can obtain bill text information and position coordinate information by acquiring a foreign language bill image and inputting the foreign language bill image into an image recognition model; then the bill text information is input into a pre-trained bill translation special model, which is trained using bills and can be used to translate bill text information and accurately translate professional field vocabulary in foreign language bill information; compared with the prior art, the scheme can reduce the semantic deviation and sentence incoherence of the translation result. Then, in order to further improve the accuracy and readability of the translation content, the Chinese bill information is input into a large language model, and the general language ability of the large language model is used to optimize the language fluency of the Chinese bill information according to the preset bill question and answer prompt words, so that the translation content can be carefully adjusted and polished, which can not only ensure accurate expression of bill information, but also make the translation content conform to the Chinese context. Then, the Chinese bill information obtained by translation is arranged according to the position coordinate information obtained above, and a Chinese bill image in the Chinese bill format is drawn, so that a complete, accurate and original layout-conformed Chinese bill image can be obtained, providing an intuitive visual experience for users. Finally, in order to facilitate subsequent financial work, a key information extraction model can efficiently and accurately extract bill key information from the Chinese bill information, ensuring the accuracy and integrity of the bill information. Because the key information extraction model is pre-trained with key word labels related to key information, it can accurately extract and identify the key information such as amount, date and item description. Through the above method, the core financial information can be accurately extracted while the foreign language bill image is accurately translated, and high-accuracy Chinese bill information is obtained. The method not only ensures the clear presentation of bill key information, but also provides a complete Chinese bill image, greatly improving the efficiency of foreign language bill reimbursement process and optimizing the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings: Figure 1 A structural schematic diagram of an original foreign language bill image provided by the prior art; Figure 2 A structural schematic diagram of a Chinese bill image directly translated by a present translation model for a bill image shown in the figure; Figure 1 Figure 3 A flowchart of a first AI-based foreign language bill image translation method provided by the embodiment of the application; ​Figure 4 A flowchart of a second foreign language bill image translation method based on artificial intelligence provided for an embodiment of the present application; Figure 5 An effect diagram of extracting bill text information, position coordinate information and confidence from a foreign language bill image provided for an embodiment of the present application; Figure 6 An effect diagram of Chinese bill information translated by a translation model provided for an embodiment of the present application; Figure 7 An effect diagram of Chinese bill information after optimization of a large language model provided for an embodiment of the present application; Figure 8 An effect diagram of optimization of a large language model through a prompt word template provided for an embodiment of the present application; Figure 9 A structural diagram of a Chinese bill image obtained by redrawing provided for an embodiment of the present application; Figure 10 A structural diagram of a key information extraction image provided for an embodiment of the present application; Figure 11 A structural diagram of an original English bill image provided for an embodiment of the present application; Figure 12 A structural diagram of a Chinese bill image after redrawing provided for an embodiment of the present application; Figure 13 A structural diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to more clearly illustrate the overall concept of the present application, the following will be described in detail with reference to the accompanying drawings.

[0019] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below. It should be noted that the embodiments of the present application and the features in each embodiment can be combined with each other without conflict.

[0020] In this application, unless otherwise expressly specified and limited, the terms "above" and "below" the second feature can refer to direct contact between the first and second features, or indirect contact between the first and second features through an intermediate medium. In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0021] The existing technology has the following drawbacks: Foreign language invoices are often highly specialized, containing not only polysemous words but also requiring the differentiation of numerous similar information points. A major challenge in foreign language invoice recognition and translation applications is processing invoices containing many domain-specific terms. Existing methods mostly employ natural language processing (NLP) models. However, NLP models tend to use general translation rules for displaying and optimizing results when performing translation tasks. This process can lead to semantic biases or awkward phrasing in the translation. Specifically, the model may incorrectly translate specialized terms into non-specialized or uncommon equivalents, resulting in decreased accuracy and even semantic errors.

[0022] like Figure 1 The original text of the foreign language invoice image shown is... Figure 2 To utilize the M2M-100 translation model for Figure 1 The image shown is a direct translation of a foreign language receipt into Chinese. The word "tip" on the foreign language receipt usually refers to a tip, not a monetary reward. Figure 2 The "hints" following the model's interpretation. In this case, the accuracy and readability of the translation are severely affected, which in turn impacts the efficiency and reliability of financial data processing. Furthermore, invoices often contain various formats for amounts, invoice dates, and delivery dates, making it easy for mis-ordered matching issues to occur when extracting this crucial information.

[0023] To overcome the aforementioned technical challenges, the following embodiments of this application provide an AI-based foreign language invoice image translation solution that can accurately translate foreign language invoices, extract core financial information, and finally obtain highly accurate Chinese translated invoices. This technology not only ensures the clear presentation of core financial information but also provides complete Chinese invoice images, greatly improving the efficiency of the foreign language invoice reimbursement process and optimizing the user experience.

[0024] To achieve the above object, see Figure 3 , Figure 3 The first foreign language bill image translation method based on artificial intelligence provided by the embodiment of the application is shown in the flowchart as shown in Figure 3 The foreign language bill image translation method based on artificial intelligence comprises the following steps: S110: obtaining a foreign language bill image, inputting the foreign language bill image into an image recognition model, and outputting bill text information and position coordinate information.

[0025] The image recognition model can use the paddle-OCR model. The paddle-OCR model is an open source OCR (Optical Character Recognition) tool library under a deep learning framework, which can support detection, recognition and direction classification of multi-language text. The embodiment of the application integrates a text recognition model and a coordinate judgment model (for example, a coordinate system transfer interface HEP) on the basis of the traditional paddle-OCR model. Thus, by scanning or taking a picture of a foreign language bill, a foreign language bill image can be obtained, and then the foreign language bill image can be input into the image recognition model to accurately recognize bill text information and position coordinate information.

[0026] Specifically, as a preferred embodiment, the step S110 in the foreign language bill image translation method comprises the following steps: S111: obtaining a recognition model training set, and classifying foreign language bill images in the recognition model training set according to language types and format types of the foreign language bill images.

[0027] First, the foreign language bill images are classified according to language types and format types. Thus, the foreign language bill images can be used to classify and train the image recognition model according to language types (for example, English, French and Russian) and format types (for example, invoices, receipts and documents). Because different language types often correspond to different countries or bill use habits, and different format types often have unique bill text information and position coordinates, classifying the foreign language bill images according to language types and format types can enable the image recognition model to accurately understand foreign language bills of different countries, different use habits and different format types, so as to quickly and accurately extract bill text information and position coordinates.

[0028] S112: data labeling the classified foreign language bill images according to the distribution positions of key information in the foreign language bill images and the types of key information, and using a clustering algorithm to generate a clustering anchor box corresponding to the key information in the foreign language bill images.

[0029] The data labeling is performed on the foreign-invoice images according to the distribution positions of the key information in the foreign-invoice images and the types of the key information (for example, time type, symbol type, and character type). In this way, the image recognition model can be more sensitive to the key information, so as to reduce the false detection and missed detection of the key information by the image recognition model. In this way, the key information in the classified foreign-invoice images is labeled, and the clustering anchor box is generated in the foreign-invoice images by using the clustering algorithm. In this way, the image recognition model can accurately locate the positions of the key information in the training process, so as to improve the recognition accuracy of the key information by the image recognition model and accurately extract the coordinate position information of the key information. The key information is in a foreign language form.

[0030] S113: The image recognition model is trained by using the foreign-invoice images that are labeled and contain the clustering anchor box and in combination with the attention mechanism.

[0031] By using the data labeling (for example, the label of the labeling type) and the clustering anchor box, the image recognition model can accurately locate the invoice character information in the foreign-invoice images, for example, the key information. In combination with the attention mechanism, the image recognition model can improve the sensitivity to the key information, so as to accurately extract the key information from the foreign-invoice images and improve the recognition accuracy of the image recognition model.

[0032] The image recognition model is trained by using the recognition model training set containing a large number of foreign-invoice images. After the training, the image recognition model can accurately recognize the invoice character information and the position coordinate information in the foreign-invoice images.

[0033] S114: When the training of the image recognition model is completed, the obtained foreign-invoice images are input into the image recognition model.

[0034] S115: The invoice character information, the position coordinate information, and the confidence are extracted from the foreign-invoice images by using the image recognition model. The invoice character information includes foreign content.

[0035] As shown in the specific embodiment, Figure 5 The foreign information extracted from the foreign-invoice images by using the image recognition model includes the position coordinate, the foreign content, and the confidence.

[0036] The foreign-invoice images are input into the image recognition model, the character content in the foreign-invoice images is accurately recognized, and the key information such as the specific position coordinate of each character is obtained. This step is the basis of the entire foreign-invoice image translation method, which ensures the accuracy and efficiency of the subsequent result processing.

[0037] Figure 3 The technical scheme provided by the embodiment shown further comprises, after the step of inputting the foreign bill image into the image recognition model and outputting bill text information and position coordinate information: S120: inputting the bill text information into a pre-trained bill translation special model to obtain Chinese bill information through translation.

[0038] Inputting the bill text information output by the image recognition model into the pre-trained bill translation special model can accurately translate foreign language content into Chinese. Because the bill translation special model is different from a general language translation model, the model is pre-trained using bill text information, and thus can accurately translate bill text information. In addition, the bill translation special model combines a pre-set Chinese bill corpus and special translation rules to translate bill text information, which can accurately solve the problem of semantic deviation or incoherent sentences in the translation result that may be caused by the natural language processing model in the prior art when performing a translation task, which tends to use general translation rules to display and optimize foreign language bills containing numerous field-specific terms. Through the above method, professional terms can be translated into corresponding Chinese words, thereby improving the accuracy of the translation result and reducing semantic errors.

[0039] In addition, as a preferred embodiment, the step S120 of inputting the bill text information into the pre-trained bill translation special model to obtain Chinese bill information through translation in the foreign bill image translation method comprises: S121: determining the data type of the bill text information according to the language type and format type of the foreign bill image and in combination with the position coordinate information.

[0040] Here, the data type of the bill text information includes time, name, address, label, and amount, etc. Using the language type and format type can determine the arrangement position and format type of each bill text information in the foreign bill image, so that in combination with the position coordinate information, the data type of the bill text information can be further determined, so that the bill translation special model can be trained using the bill text information according to the data type, and the position coordinates in the Chinese format can be restored in combination with the above data type and position coordinate information.

[0041] S122: classifying the bill text information to obtain a translation model training set containing bill text information of multiple data types.

[0042] Classifying the bill text information according to the above data type, the translation model training set obtained can be used to train the translation model, so that the translation model can accurately locate the bill text information of different data types and coordinate positions.

[0043] S123: input the translation model training set into the preset translation model, train the preset translation model in combination with the preset Chinese bill corpus to obtain a bill translation special model; wherein the preset Chinese bill corpus contains professional Chinese bill corpus corresponding to the data type.

[0044] Because the bill text information in the translation model training set is classified according to the data type, the preset translation model can be trained for bill text information of different data types respectively, so as to obtain a bill translation special model, which can accurately locate bill text information of different data types and coordinate positions, and improve the translation accuracy of bill text information. The untrained preset translation model herein can use the M2M-100 translation model.

[0045] As a preferred embodiment, the step S123 of training the preset translation model in combination with the preset Chinese bill corpus to obtain a bill translation special model comprises: S1231: construct a special semantic space using the Chinese bill corpus, and train the preset translation model using the Chinese bill corpus, so that the preset translation model learns the bill translation rules corresponding to the Chinese bill corpus.

[0046] S1232: control the preset translation model to map the bill text information into professional word vectors in the special semantic space according to the bill translation rules.

[0047] S1233: combine all professional word vectors corresponding to the bill text information, convert all professional word vectors into Chinese text, and obtain Chinese bill information.

[0048] The technical scheme provided by the embodiments of the present application uses the preset translation model to learn the bill translation rules corresponding to the special semantic space of the field, so as to map the professional terms in the bill text information into professional word vectors in the special semantic space according to the bill translation rules, which can accurately understand the Chinese meaning corresponding to the professional terms in the bill text, thereby improving the translation accuracy of professional terms. In summary, the technical scheme provided by the embodiments of the present application accurately understands the Chinese text meaning corresponding to the bill text information according to the context and context of the bill text information by combining the Chinese bill corpus and the bill translation rules, and realizes accurate translation of professional bill text information.

[0049] S124: input the bill text information into the bill translation special model, and output the Chinese bill information and the Bool value corresponding to the bill text information. After outputting the Chinese bill information and the Bool value using the bill special translation model, the Chinese bill image in Chinese format can be further redrawn and restored in combination with the position coordinate information and the confidence, and the accuracy of model translation can be improved.

[0050] The translation model optimized by the large model is used to obtain a bill translation special model, and bill text information input can obtain Figure 6 The translated position coordinates, Chinese bill information, confidence, and Bool value of whether to translate are shown in FIG. 6. Based on the translation content, the Chinese bill image shown in FIG. 7 can be redrawn, thereby ensuring accurate information transmission and visual consistency. In summary, the technical scheme provided by the embodiments of the present application can transmit the text content and position information output by the image recognition model into the translation model according to the language type and format type of the foreign bill image, and can accurately translate the foreign content into Chinese. Figure 9

[0051] In addition, the Chinese bill information translated using the bill translation special model may have a certain degree of stiffness and directness, which does not conform to the Chinese context. As shown in FIG. 8, the translated content "100% of the money is for the driver" conveys the correct meaning, but whether the expression is stiff leads to poor readability and accuracy of the translation content. In order to further improve the accuracy and readability of the translation content, after obtaining the Chinese bill information, the translated Chinese bill information needs to be input into the large language model for optimization to polish the translation content in terms of language fluency with the help of the large language model. Figure 6

[0052] The above step S120: inputting the bill text information into the pre-trained bill translation special model to translate the Chinese bill information further includes: S130: inputting the Chinese bill information into the large language model and optimizing the language fluency of the Chinese bill information according to the preset bill question and answer prompt words.

[0053] In order to further improve the accuracy and readability of the translation content, when inputting the Chinese bill information into the large language model for polishing, the bill question and answer prompt words are written to polish the translation answer of the Chinese bill information by the large language model, which can successfully optimize the translation effect and perfectly adapt to the expression habits of the Chinese context.

[0054] Specifically, as a preferred embodiment, the step S130: inputting the Chinese bill information into the large language model and optimizing the language fluency of the Chinese bill information according to the preset bill question and answer prompt words includes: S131: writing a plurality of bill question and answer prompt words according to Chinese expression habits and constructing a bill prompt word template using the plurality of bill question and answer prompt words. The bill question and answer prompt words include professional terms, noun concepts, Chinese language features, and special attention of the bill, etc.

[0055] ​​S132: input the Chinese bill information into the large language model, use the question and answer prompt words in the bill prompt word template to optimize the output of the large language model, and obtain the Chinese bill information with optimized language fluency. Here, the large language model is used to optimize the Chinese bill information in combination with the question and answer prompt words. Specifically, the semantic accuracy of the polysemous professional terms can be optimized, and the professional terms of the foreign language abbreviations can be corrected, thereby reducing the challenges brought by the frequent use of professional field abbreviations in foreign language bills to translation.

[0056] Specifically referring to Figure 7 As Figure 7 shown, the data obtained after original translation, such as "terms and conditions", "Cambridge", "news about football weapons", "the car in front" and "you can send a check to East Repair Company", are polished by question and answer prompt words, and the translated content is as follows: "East Repair Company 1912 Harvest Lane New York, NY 12210 Bill to: John Smith ……" After translation, the language habits in Chinese context are completely consistent.

[0057] In order to further improve the accuracy and readability of the translated content, the embodiments of the present application send the translated Chinese information into a large language model for optimization. Through the carefully designed bill question and answer prompt words, the large model can make detailed adjustments and polishing on the translated content, and ensure that the expression of the bill information is accurate and consistent with the Chinese context. Specifically, through the prompt word template, the optimized results obtained by optimizing the large language model are as shown in Figure 8 The large language model can exhibit its significant advantages in the field of language processing, especially in text understanding and text optimization. The embodiments of the present application utilize the powerful ability of the large language model to make detailed polishing on the translation results. The answer is standardized and the translated answer is polished by writing question and answer prompt words. The translation effect can be successfully optimized through the prompt word template.

[0058] In addition, in order to further improve the accuracy and readability of the translated content, the embodiments of the present application combine the large language model and the bill translation special model to construct the bill translation-large language model. The Chinese bill information output by the bill translation special model is used as the predicted value, and the optimized Chinese bill information output by the large language model is used as the true value. The output of the bill translation special model is further optimized by rolling back to the bill translation special model. At the same time, the loss function of the bill translation special model is designed. The loss function is the error function between the above true value and predicted value. When the loss function converges, the bill translation special model is successfully trained. In this way, the output of the large language model is rolled back to the bill translation special model, which can further improve the accuracy and readability of the output of the bill translation special model.

[0059] Figure 3The technical solution provided by the illustrated embodiment, in step S130: input the Chinese bill information into the large language model, and after the language fluency optimization of the Chinese bill information according to the preset bill question and answer prompt word, further includes: S140: using the position coordinate information, arranging and combining the optimized Chinese bill information according to the Chinese bill format, and drawing a Chinese bill image in the Chinese bill format.

[0060] The embodiment of the present application uses the position coordinate information obtained in the image recognition stage to redraw the Chinese bill image according to the Chinese bill format. In this way, a complete, accurate and original text layout compliant Chinese bill image can be obtained, providing users with an intuitive visual experience.

[0061] Specifically, as a preferred embodiment, the step S140: using the position coordinate information, arranging and combining the optimized Chinese bill information according to the Chinese bill format, and drawing a Chinese bill image; specifically includes: S141: generating a Chinese bill framework according to the Chinese bill format, wherein the Chinese bill framework includes the data type corresponding to the Chinese bill information and the Chinese filling area.

[0062] S142: mapping the Chinese bill information to the data type in the Chinese bill framework according to the corresponding relationship between the position coordinate information and the Chinese filling area.

[0063] Foreign bill formats are diverse, and key information such as amount, date and item description may be presented in various forms, which increases the complexity of information extraction. In attempting to extract key information from bills, existing general natural language processing models often make errors, such as incorrect amount recognition, failed date format conversion or incomplete item description extraction, etc.

[0064] Finally, in order to facilitate the subsequent financial work, key information extraction is performed on the optimized translation content. Through the key information extraction model, key information such as date, amount and purpose can be efficiently extracted from the bill, ensuring the accuracy and integrity of the bill information and providing strong data support for enterprises. Through this series of processes, not only the rapid translation and image conversion of foreign bills are realized, but also convenient and efficient services for enterprise financial work are provided.

[0065] Figure 3 The technical solution provided by the illustrated embodiment, after drawing the Chinese bill image, further includes: S150: extracting the bill key information from the Chinese bill information using a key information extraction model, wherein the key information extraction model is pre-trained by keyword labeling. The key information extraction model can be trained using the UIE model under the paddleNLP framework to obtain an information extraction model specially used for bill key information extraction.

[0066] The technical scheme provided by the embodiments of the present application can accurately restore the Chinese bill image by generating a Chinese bill framework according to the Chinese bill format, mapping the Chinese bill to the region according to the position coordinate information and the Chinese filling region obtained above, and combining with the data types such as time, cost and visa information. In summary, Figure 11 The original foreign bill image shown in the figure can finally generate Figure 12 The translated bill image shown in the figure is fluent and accurate.

[0067] Specifically, as a preferred embodiment, in the foreign bill image translation method, step S150: extracting the bill key information from the Chinese bill information using a key information extraction model, specifically includes: S151: labeling the bill key information in the Chinese bill information using a label labeling tool to obtain an extraction model training set.

[0068] S152: training the key information extraction model using the extraction model training set until the loss function of the key information extraction model converges.

[0069] S153: inputting the optimized Chinese bill information into the key information extraction model.

[0070] S154: extracting the bill key information from the Chinese bill information using the key information extraction module.

[0071] The technical scheme provided by the embodiments of the present application can ensure the accuracy and integrity of the bill information by labeling the bill key information in the Chinese bill information using a label labeling tool to obtain an extraction model training set and training the key information extraction model, thereby providing strong data support for enterprises. Through this series of processes, not only the rapid translation and image conversion of foreign bills are realized, but also convenient and efficient services are provided for enterprise financial work.

[0072] As a preferred embodiment, in the foreign bill image translation method, the step S150 of extracting the bill key information from the Chinese bill information using a key information extraction model includes: S155: labeling the key information of the Chinese bill sample using a Doccano labeling tool to obtain keyword labels. S156: Set the initial learning rate and loss function of the key information extraction model, input the Chinese bill sample containing the keyword label into the key information extraction model for multi-round training until the loss function converges; S157: Input the Chinese bill information into the key information extraction model to extract the bill key information.

[0073] In the technical scheme provided by the embodiments of the present application, in the field of bill extraction, key information such as amount and date needs to be extracted for archiving and approval, for example, Figure 10 Therefore, after obtaining the translated bill result, the Chinese bill information needs to be extracted by the key information extraction model. The embodiments of the present application use the UIE model under the PaddleNLP framework for extraction, but the extraction content of the basic extraction model is not complete. The date content can be completely extracted, but it is difficult to recognize complex amount information. Therefore, the embodiments first perform data labeling and training on the bill. Specifically, the data set is labeled and processed for the "date" and "amount" information in the bill by using the Doccano labeling tool to obtain labeled data; then the initial learning rate is set; the number of data samples input each time the neural network is trained is 32; the maximum token number of the text input into the information extraction model is 512; the information extraction model obtained after 100 rounds of training is applied in the key information extraction process to obtain readable and ordered key information, and the complete date and total amount can be extracted.

[0074] Figure 3 The technical scheme provided by the embodiments shown further includes: S160: Display the Chinese bill image and the bill key information.

[0075] The foreign language bill image translation method based on artificial intelligence provided by the embodiment of the application can obtain bill text information and position coordinate information by acquiring a foreign language bill image and inputting the foreign language bill image into an image recognition model; then the bill text information is input into a pre-trained bill translation special model, the bill translation special model is trained using bills and can be used to translate bill text information and accurately translate professional field vocabulary in foreign language bill information; compared with the prior art of using a large model or general translation rules to translate foreign language bills, the method can reduce the semantic deviation and incoherent problems of the translation results. Then, in order to further improve the accuracy and readability of the translation content, the Chinese bill information is input into a large language model, and the general language ability of the large language model is used to optimize the language fluency of the Chinese bill information according to the preset bill question and answer prompt words, so that the translation content can be carefully adjusted and polished, which can not only ensure accurate expression of bill information, but also make the translation content conform to the Chinese context. Then, the Chinese bill information obtained by translation is arranged according to the position coordinate information obtained above, and a Chinese bill image in the Chinese bill format is drawn, so that a complete, accurate and original layout conforming Chinese bill image can be obtained, providing an intuitive visual experience for the user. Finally, in order to subsequent financial work, the key information extraction model can efficiently and accurately extract bill key information from the Chinese bill information, ensuring the accuracy and integrity of the bill information, because the key information extraction model is pre-trained with key word labels related to key information, so it can accurately extract and identify the key information such as amount, date and item description. Through the above method, the core financial information can be accurately extracted while the foreign language bill image is accurately translated, and high-accuracy Chinese bill information is obtained. The method not only ensures the clear presentation of bill key information, but also provides a complete Chinese bill image, greatly improving the efficiency of foreign language bill reimbursement process and optimizing the user experience.

[0076] In addition, referring to Figure 4 , Figure 4 The flowchart of the second foreign language bill image translation method based on artificial intelligence provided by the embodiment of the application is shown in FIG. 2. Figure 4 As shown in the figure, the foreign language bill image translation method comprises the following steps. S210: foreign language invoice image input.

[0077] S220: image content recognition and information position marking.

[0078] S230: image recognition content input into a translation model to obtain Chinese information.

[0079] S240: Chinese information import into a large model and use prompt words to optimize translation content.

[0080] S250: key information extraction on the optimized translation content.

[0081] S260: Chinese invoice image display after translation.

[0082] In addition, the product embodiments provided by the following embodiments of the present application have the same beneficial effects as the foreign language invoice image translation method based on artificial intelligence provided by the above embodiments, and other technical features in the product embodiments are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0083] Reference is made to Figure 13 , which shows a structural schematic diagram of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application can include but is not limited to a mobile terminal and / or a fixed terminal. Figure 13 The device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0084] Reference is made to Figure 13 , which shows a structural schematic diagram of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application can include but is not limited to a mobile terminal and / or a fixed terminal. Figure 13 The device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0085] As Figure 13 shown, the electronic device can include a processing device 1001 in the form of a central processing unit and / or a graphics processing unit, which can perform various appropriate actions and processes according to programs stored in a ROM 1002 or loaded from a storage device 1003 to a RAM 1004. Various programs and data required for the operation of the above electronic device are also stored in the RAM 1004. The processing device 1001, the ROM 1002 and the RAM 1004 are connected to each other through a bus 1005. An input / output interface 1006 is also connected to the bus 1005, and the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer and / or a gyroscope; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker and / or a vibrator; storage devices 1003 including, for example, a magnetic tape and / or a hard disk; and communication devices 1009. The communication devices 1009 can perform wireless or wired communication between the above electronic device and other devices to exchange data. Although a model construction device with various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0086] In particular, according to the embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the artificial intelligence-based foreign language bill image translation method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the artificial intelligence-based foreign language bill image translation method of the embodiments of the present disclosure are executed.

[0087] The present disclosure provides a computer readable storage medium having stored thereon computer readable program instructions, i.e., the above-mentioned computer program, for executing the artificial intelligence-based foreign language bill image translation method in the above-mentioned embodiments.

[0088] The above-mentioned computer readable storage medium carries one or more programs, which, when executed by a model construction device, can be written in one or more programming languages or combinations thereof for executing computer program codes for performing the operations of the present disclosure, the programming languages including object-oriented programming languages such as Java, Smalltalk or C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on a user computer, partially on a user computer, as a separate software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network LAN (Local Area Network) or a wide area network WAN, or can be connected to an external computer, such as an Internet service provider, through the Internet.

[0089] The computer program product of the present application can be a computer program embodied on a non-transitory computer readable medium. When the program runs on a computer, the flowchart and / or block diagram in the flowchart and / or block diagram can be implemented.

[0090] The modules involved in the embodiments of the present application can be implemented in software or hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0091] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments.

[0092] The above only describes the embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An artificial intelligence-based foreign language bill image translation method, characterized by, The method comprises the following steps: obtaining an image of a foreign language bill, inputting the image of the foreign language bill into an image recognition model, and outputting bill text information and position coordinate information; inputting the bill text information into a pre-trained bill translation special model to obtain Chinese bill information; inputting the Chinese bill information into a large language model, and optimizing the language fluency of the Chinese bill information according to a preset bill question and answer prompt word; using the position coordinate information, arranging and combining the optimized Chinese bill information according to a Chinese bill format, and drawing a Chinese bill image in a Chinese bill format; extracting bill key information from the Chinese bill information using a key information extraction model, wherein the key information extraction model is pre-trained by key word labels; displaying the Chinese bill image and the bill key information.

2. The method of claim 1, wherein, The step of inputting the image of the foreign language bill into the image recognition model and outputting the bill text information and the position coordinate information comprises the following steps: obtaining a recognition model training set, classifying the foreign language bill images in the recognition model training set according to the language type and format type of the foreign language bill images; performing data labeling on the classified foreign language bill images according to the distribution position of the key information in the foreign language bill images and the key information type, and generating a clustering anchor box corresponding to the key information in the foreign language bill images using a clustering algorithm; training the image recognition model using the data labeled foreign language bill images containing the clustering anchor box and combining the attention mechanism; when the image recognition model is trained, inputting the obtained foreign language bill image into the image recognition model; extracting bill text information, position coordinate information and confidence from the foreign language bill image using the image recognition model.

3. The method of claim 1 or 2, wherein, The step of inputting the bill text information into the pre-trained bill translation special model to obtain Chinese bill information comprises the following steps: determining the data type of the bill text information according to the language type and format type of the foreign language bill image and combining the position coordinate information; classifying the bill text information to obtain a translation model training set containing bill text information of multiple data types; inputting the translation model training set into a preset translation model, training the preset translation model by combining a preset Chinese bill corpus to obtain the bill translation special model; wherein the preset Chinese bill corpus contains professional Chinese bill corpus corresponding to the data type; inputting the bill text information into the bill translation special model to output Chinese bill information corresponding to the bill text information and a Bool value.

4. The method of claim 3, wherein, The step of training the preset translation model by combining the preset Chinese bill corpus to obtain the bill translation special model comprises the following steps: constructing a special semantic space using the Chinese bill corpus, and training the preset translation model using the Chinese bill corpus, so that the preset translation model learns the bill translation rules corresponding to the Chinese bill corpus; The preset translation model is controlled to map the bill character information into professional word vectors in the professional semantic space according to the bill translation rule; All professional word vectors corresponding to the bill character information are combined, and the all professional word vectors are converted into Chinese text to obtain the Chinese bill information.

5. The method of claim 1, wherein, The step of inputting the Chinese bill information into the large language model and optimizing the Chinese bill information in terms of language fluency according to a preset bill question and answer prompt word includes: A plurality of bill question and answer prompt words are written according to Chinese expression habits, and a bill prompt word template is constructed using the plurality of bill question and answer prompt words; The Chinese bill information is input into the large language model, and the output of the large language model is optimized using the question and answer prompt words in the bill prompt word template to obtain the Chinese bill information optimized in terms of language fluency.

6. The method of claim 1, wherein, The step of arranging and combining the Chinese bill information optimized according to the Chinese bill format to draw a Chinese bill image includes: A Chinese bill framework is generated according to the Chinese bill format, wherein the Chinese bill framework includes data types and Chinese filling areas corresponding to the Chinese bill information; The Chinese bill information is mapped to the data types in the Chinese bill framework according to the correspondence between the position coordinate information and the Chinese filling areas.

7. The method of claim 1, wherein, The step of extracting bill key information from the Chinese bill information using a key information extraction model includes: A label annotation tool is used to label the bill key information in the Chinese bill information to obtain an extraction model training set; The key information extraction model is trained using the extraction model training set until the loss function of the key information extraction model converges; The optimized Chinese bill information is input into the key information extraction model; The key information extraction module is used to extract the bill key information from the Chinese bill information.

8. The method of claim 1, wherein, The step of extracting bill key information from the Chinese bill information using a key information extraction model includes: A Doccano annotation tool is used to annotate key information of a Chinese bill sample to obtain a keyword label; An initial learning rate and a loss function of a key information extraction model are set, and the Chinese bill sample containing the keyword label is input into the key information extraction model for multiple rounds of training until the loss function converges; The Chinese bill information is input into the key information extraction model to extract the bill key information.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the artificial intelligence-based foreign bill image translation method according to any one of claims 1 to 8 when executing the program.

10. A computer storage medium having stored thereon a computer program, characterized in that The computer program implements the artificial intelligence-based foreign bill image translation method according to any one of claims 1 to 8 when executed.

Images