Text data processing method and device, question and answer information processing method and device and electronic equipment
By receiving and adjusting text and model processing, the problem of inaccurate table information extraction in the existing technology is solved, and more efficient and accurate table data generation and large model question and answer support are achieved.
Patent Information
- Application Number
- CN202510812524.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-16
AI Technical Summary
Existing technologies tend to lose structural and style details when extracting table information from documents, making it difficult for large models to accurately understand and apply them. In particular, for complex tables and nested structures, the recognition accuracy is low and the generated structured information is not directly suitable for downstream applications.
By receiving the adjustment text provided by the user or model, the descriptive text of the selected object in the rich text data block to be processed is determined and the processed rich text data block is generated. The target model is used for image processing and data adjustment to ensure the accuracy and consistency of the table information.
It improves the accuracy and consistency of table information, reduces labor costs, improves data generation efficiency, and provides large models with richer corpus to support more accurate question and answer generation.
Smart Images
Figure CN120653816A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence technology, particularly image processing, large models, and retrieval augmented generation (RAG), and is applicable to intelligent question-and-answer scenarios. More specifically, this disclosure provides a text data processing method, a question-and-answer information processing method, apparatus, electronic device, and storage medium. Background Art
[0002] With the development of artificial intelligence technology, the application of large models is increasing. In order to enable large models to accurately answer questions in different fields, different knowledge bases can be built for different fields so that the large models can generate answers based on the data in the knowledge base. Summary of the Invention
[0003] The present disclosure provides a text data processing method, a question and answer information processing method, an apparatus, a device, and a storage medium.
[0004] According to one aspect of the present disclosure, a text data processing method is provided, the method comprising: in response to receiving an adjustment text for a rich-text data block to be processed, determining at least one descriptive text for at least one selected object in the adjusted text, wherein the rich-text data block to be processed is obtained based on an image to be processed, the rich-text data block to be processed is used to represent at least one candidate object, and the adjustment text is used to indicate at least one selected object among the at least one candidate object; generating a processed rich-text data block for the adjusted text based on the at least one descriptive text for the at least one selected object.
[0005] According to another aspect of the present disclosure, a question-and-answer information processing method is provided, the method comprising: determining context information for the question information from a knowledge base based on received question information; generating answer information for the question information based on the context information, wherein the knowledge base comprises a plurality of target rich-text data blocks determined from a plurality of images to be processed, and the target rich-text data blocks are determined by the following operations: in response to receiving an adjustment text for the rich-text data block to be processed, determining at least one descriptive text for at least one selected object in the adjustment text, wherein the rich-text data block to be processed is obtained based on the image to be processed, the rich-text data block to be processed is used to represent at least one candidate object, and the adjustment text is used to indicate at least one selected object among the at least one candidate object; generating a processed rich-text data block for the adjustment text based on the at least one descriptive text for the at least one selected object; and in response to receiving a confirmation text for the processed rich-text data block, determining the processed rich-text data block as the target rich-text data block of the image to be processed.
[0006] According to another aspect of the present disclosure, a text data processing device is provided, which includes: a first determination module for determining at least one descriptive text for at least one selected object in the adjusted text in response to receiving an adjusted text for a rich-text data block to be processed, wherein the rich-text data block to be processed is obtained based on an image to be processed, the rich-text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object among the at least one candidate object; and a first generation module for generating a processed rich-text data block for the adjusted text based on the at least one descriptive text for the at least one selected object.
[0007] According to another aspect of the present disclosure, a question-and-answer information processing device is provided, which includes: a first determination module for determining, in response to receiving an adjustment text for a rich-text data block to be processed, at least one descriptive text for at least one selected object in the adjusted text, wherein the rich-text data block to be processed is obtained based on an image to be processed, the rich-text data block to be processed is used to represent at least one candidate object, and the adjustment text is used to indicate at least one selected object among the at least one candidate object; a first generation module for generating, based on at least one descriptive text for the at least one selected object, a processed rich-text data block for the adjustment text; and a second determination module for determining, in response to receiving a confirmation text for the processed rich-text data block, the processed rich-text data block as a target rich-text data block of the image to be processed.
[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided according to the present disclosure.
[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.
[0010] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the method provided according to the present disclosure when executed by a processor.
[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.
[0013] Figure 1 is a schematic diagram of an exemplary system architecture to which a text data processing method, a question-and-answer information processing method, and an apparatus can be applied according to an embodiment of the present disclosure;
[0014] Figure 2 is a flowchart of a text data processing method according to an embodiment of the present disclosure;
[0015] Figure 3A is a schematic diagram of an image to be processed according to an embodiment of the present disclosure;
[0016] Figure 3B is a schematic diagram of a rendered image according to one embodiment of the present disclosure;
[0017] Figure 4 is a flowchart of a question-and-answer information processing method according to an embodiment of the present disclosure;
[0018] Figure 5 is a block diagram of a text data generating apparatus according to an embodiment of the present disclosure;
[0019] Figure 6 is a block diagram of a question and answer information processing apparatus according to another embodiment of the present disclosure; and
[0020] Figure 7 4 is a block diagram of an electronic device to which at least one of a text data processing method and a question-and-answer information processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0021] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0022] The large model can be a Large Language Model (LLM). Various documents are important data sources for building knowledge bases. Documents can include images and tables. Optical Character Recognition (OCR) technology can be used to extract text content from images, but it is difficult to obtain information such as structure and style. Based on image processing and preset extraction rules, the row and column structure of a table can be identified by detecting lines, borders, etc. Machine learning models or simple deep learning models can also be used to detect table areas and identify basic structures. Some general multimodal large models can be used to understand image content, but it is difficult to generate refined structure and style based on table images.
[0023] When extracting and converting table information from documents based on methods such as optical character recognition, machine learning models, simple deep learning models, or general multimodal large models, the structure and style details of the table are easily lost, resulting in incomplete information and making it difficult for large models to efficiently understand and apply it. For complex tables with merged rows or columns, the visual style information such as the merged relationship between rows and columns, the color, font, and alignment within the cells are more likely to be lost. If the table has no obvious separators, has complex row and column merges, or contains nested structures, the recognition accuracy is also low. Rich visual style information such as the cell background color, text color, font, font size, and alignment is almost completely lost during the conversion process. In addition, when extracting and converting table information, the structured information generated by machine learning models, simple deep learning models, or general multimodal large models is often not directly applicable to downstream applications and requires a lot of post-processing.
[0024] In order to fully utilize the chart information in the document to generate highly accurate and in-depth answers, the present disclosure provides a text data processing method, which will be described below.
[0025] Figure 1 This is a schematic diagram of an exemplary system architecture to which the text data processing method, question and answer information processing method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.
[0026] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0027] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers, etc.
[0028] Server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal device.
[0029] It should be noted that the text data processing method and question-and-answer information processing method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the text data processing apparatus and question-and-answer information processing apparatus provided in the embodiments of the present disclosure can generally be located in the server 105. The text data processing method and question-and-answer information processing method provided in the embodiments of the present disclosure can also be executed by the terminal devices 101, 102, and 103. Accordingly, the text data processing method and question-and-answer information processing apparatus provided in the embodiments of the present disclosure can also be located in the terminal devices 101, 102, and 103. The text data processing method and question-and-answer information processing method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, and 103 and / or the server 105. Accordingly, the text data processing apparatus and question-and-answer information processing apparatus provided in the embodiments of the present disclosure can also be located in a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, and 103 and / or the server 105.
[0030] It can be understood that the above describes the system architecture of the present disclosure.
[0031] Figure 2 is a flowchart of a text data processing method according to an embodiment of the present disclosure.
[0032] like Figure 2As shown, the method 200 may include operations S210 to S220.
[0033] In operation S210 , in response to receiving an adjusted text for a rich text data block to be processed, at least one description text for at least one selected object in the adjusted text is determined.
[0034] In the embodiment of the present disclosure, the rich text data block may be a code block based on Hypertext Markup Language (HTML), and the code block may include one or more lines of code.
[0035] In the embodiment of the present disclosure, the rich text data block to be processed may be obtained based on the image to be processed. The image to be processed may be a table image in the above-mentioned document, or an image such as a bar chart or line chart with structural information.
[0036] In the disclosed embodiments, a block of rich text data to be processed may represent at least one candidate object. For example, if the block of rich text data to be processed is obtained from a table image, rows, columns, and cells in the table may serve as candidate objects. For another example, if the block of rich text data to be processed is obtained from a bar chart, the coordinate axes and columns in the bar chart may serve as candidate objects.
[0037] In an embodiment of the present disclosure, adjustment text is used to indicate at least one selected object from at least one candidate object. For example, the object mentioned in the adjustment text from at least one candidate object can be considered the selected object. Taking a table image as an example, the adjustment text can be "The background of the cell in the second row and second column should be red." Based on this adjustment text, it can be determined that the selected object is the cell in the second row and second column. The description text can be "The background should be red."
[0038] In operation S220, a processed rich text data block for adjusting text is generated based on at least one description text for at least one selected object.
[0039] In the embodiment of the present disclosure, a processed rich text data block different from the to-be-processed rich text data block may be regenerated to correct problems or errors in the selected object indicated by the description text.
[0040] Through the disclosed embodiments, adjustment text can be received for a rich text data block of an image to be processed, allowing adjustments to be made based on feedback regarding the rich text data block to be processed, thereby improving the accuracy of the rich text data block. The adjustment text, which includes descriptive text for a selected object, can accurately and efficiently adjust elements in the image (e.g., rows, columns, and cells), resulting in more accurate rich text data block generation with less time cost.
[0041] It can be understood that the above describes the method of the present disclosure, and the following describes the image to be processed of the present disclosure.
[0042] Figure 3A is a schematic diagram of an image to be processed according to an embodiment of the present disclosure.
[0043] like Figure 3A As shown, the image to be processed imgae30 can be a table image to be processed. The target model can be used to generate an initial rich text data block based on the image to be processed as the rich text data block to be processed. The target model can be a multimodal model with a parameter amount of 10 billion (10B), which can parse the input image to be processed, identify the text, lines, areas, colors and other information in the image to be processed, and also obtain the overall layout and structural relationship of the table. Through the embodiment of the present disclosure, rich text data is extracted from the table image to be processed and adjusted, which can effectively extract the information in the table and provide more and richer corpus for the large model.
[0044] In addition, if Figure 3A As shown, the image to be processed image30 includes a plurality of main cell objects, including a main cell object td322, whose background is red.
[0045] For example, based on the image to be processed, the rich text data block html_1 to be processed determined by the target model can be:
[0046]
[0047]
[0048]
[0049]
[0050]
[0051]
[0052]
[0053]
[0054]
[0055]
[0056]
[0057]
[0058]
[0059]
[0060]
[0061]
[0062] name The Group holds interests Gold produced from smelting (kg) Equity smelter gold (kg) Company A 100% 139,600 139,600 Company B 70% 53,415 37,390 Company C 61% 46,000 28,060 other 32,875 29,791 total 271,890 234,841
[0063] In some embodiments, the rich text data block to be processed may include at least one candidate data, and the candidate data may represent a candidate object.
[0064] The above-mentioned rich text data block html_1 to be processed includes candidate table data. The candidate table data includes the table start character , table content data and table terminator In addition, the border attribute is included in the table start character. The value of the border attribute is 1, which means that the table has a border. and table terminator The multiple lines of code between can be used as table content data. The candidate table data can represent a candidate table object.
[0065] The above-mentioned rich text data block html_1 to be processed also includes multiple candidate area data. The multiple candidate area data include candidate header area data and candidate table body area data. The candidate header area data includes a header area start character, header data and a header area end character. The multiple lines of code between the header area start character and the header area end character can be used as header data. The candidate header area data can represent a candidate header area object. The candidate table body area data includes a table body area start character, table body data and a table body area end character. The multiple lines of code between the header area start character and the header area end character can be used as table body data. The candidate table body area data can represent a candidate table body area object.
[0066] The above-mentioned rich text data block html_1 to be processed also includes multiple candidate row data. The multiple candidate row data include candidate header row data and multiple candidate table body row data. The candidate header row data can be the above-mentioned header data. The candidate header row data includes a row start character, multiple candidate header cell data and a row end character. The candidate header row data can represent a candidate header row object. The multiple candidate body row data can be the above-mentioned table body data. The candidate table body row data includes a row start character, multiple candidate body cell data and a row end character. The candidate table body row data can represent a candidate table body row object.
[0067] The above-mentioned rich text data block html_1 to be processed may also include multiple candidate cell data. The multiple candidate cell data include multiple candidate header cell data and multiple candidate body cell data. Taking the above-mentioned candidate header cell data "name" as an example, the candidate header cell data includes the header cell start symbol, the header cell content information "name" and the header cell end symbol. The candidate header cell data can represent a candidate header cell object. Taking the above-mentioned candidate body cell data "139,600" as an example, the candidate body cell data includes the body cell start symbol, the body cell content information "139,600" and the body cell end symbol.
[0068] In some embodiments, the candidate data may further include attribute information for the candidate object. The attribute information may at least indicate the visual style of the candidate object. Figure 3A As shown, the candidate table data includes the attribute information "border="1"", which can indicate that the entire table is a table with a border.
[0069] It can be understood that the above description has described the image to be processed and the rich text data block to be processed of the present disclosure, and the following description will describe the adjusted text of the present disclosure.
[0070] In some embodiments, the above method may further include: rendering the rich text data block to be processed to obtain a rendered image. Figure 3B Provide explanation.
[0071] Figure 3B is a schematic diagram of a rendered image according to one embodiment of the present disclosure.
[0072] like Figure 3B As shown, the rendered image image31 is generated by compiling and rendering the rich text data block html_1 to be processed, and can be displayed on a visual interface. Through the disclosed embodiments, the rich text data block is compiled and rendered, converting the rich text data block into data of the same modality as the image to be processed. This allows for more accurate determination of deviations in the rich text data block to provide more precise adjustment text for the model.
[0073] In some embodiments, the method may further include: obtaining adjusted text input by the user. Through the disclosed embodiments, the user provides feedback on the extracted rich text data block, which can make the adjusted rich text data block more in line with the user's needs and effectively improve the user experience.
[0074] like Figure 3A As shown in the figure, the background of the main cell object td322 is red. Figure 3B As shown, the background of the candidate cell object td322' is the default white. The adjustment text input by the user may be "the background of the cell in the second row and second column should be red."
[0075] Next, operation S210 can be performed. In response to receiving the adjusted text for the rich text data block to be processed, html_1, the target model can determine the descriptive text for the selected object in the adjusted text. For the adjusted text "The background of the cell in the second row and second column should be red," the selected object can be determined to be the candidate subject cell object td322', and the descriptive text can also be determined to be "The background should be red." The selected data for the selected object can be the candidate subject cell data "139,600."
[0076] In some embodiments, the description text may describe: the deviation between the selected data for the selected object and the baseline data for the selected object. For example, taking the description text "the background should be red" as an example, the baseline data of the subject cell object td322' to be processed may indicate that the cell background is red. The description text describes that there is a deviation in the cell background, and the way to eliminate or reduce the deviation may be to adjust the background color to red. It can be understood that in the case where the adjustment text is user input, the baseline data may be obtained by the user from the image to be processed. Through the embodiment of the present disclosure, the description text describing the deviation is determined from the adjustment text, and an accurate rich text data block can be generated to improve the adjustment efficiency.
[0077] Next, in some embodiments of the above-described operation S220, generating a processed rich-text data block for adjusting text based on at least one descriptive text for at least one selected object includes: generating processed selected data using a target model based on the descriptive text for the selected object to reduce the deviation between the processed selected data and the baseline data. Through the embodiments of the present disclosure, when generating the processed rich-text data block, targeted data generation based on the descriptive text can effectively improve the rich-text data block, increase adjustment efficiency, and more efficiently obtain accurate rich-text data blocks.
[0078] In some embodiments, the candidate data may be generated by the target model based on the first inference path and the first internal representation data. The target model may perform inference based on different inference paths. The internal representation data may include some static parameters and some dynamic parameters of the model. The static parameters remain unchanged after the target model is trained. The dynamic parameters may change during each inference to introduce randomness.
[0079] In some embodiments, generating a processed selection for a selected object using the target model includes at least one of the following operations: generating processed selection data based on the second reasoning path; generating first processed selection data based on the second internal representation data. For example, to generate a processed rich text data block, the generated data may be generated based on the image to be processed and at least one descriptive text based on at least one of the second reasoning path and the second internal representation data. The dynamic parameters in the second internal representation data differ from the dynamic parameters in the first internal representation data.
[0080] For example, the processed rich text data block html_2 generated by the target model can be:
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
[0089]
[0090]
[0091]
[0092]
[0093]
[0094]
[0095]
[0096]
[0097] name The Group holds interests Gold produced from smelting (kg) Equity smelter gold (kg) Company A 100% 139,600 139,600 Company B 70% 53,415 37,390 Company C 61% 46,000 28,060 other 32,875 29,791 total 271,890 234,841
[0098] In some embodiments, the processed rich text data block includes at least one processed data. The at least one processed data may include processed table data, processed table area data, processed row data, and processed cell data. It will be understood that the above description of candidate table data, candidate table area data, candidate row data, and candidate cell data also applies to processed table data, processed table area data, processed row data, and processed cell data.
[0099] In some embodiments, at least one processed data includes at least one processed selected data. The processed selected data is used to represent the processed selected object corresponding to the selected object. Unlike the unprocessed rich text data block html_1, the processed rich text data block html_2 includes processed body cell data "139,600," which corresponds to the candidate body cell td322' and is the processed selected data. Unlike the unprocessed body cell data "139,600," the processed body cell data "139,600" also includes background attribute information "style="background-color: red;"." The background of the processed body cell object represented by this processed body cell data is red. It can be understood that the processed body cell object represented by this processed body cell data can be used as the processed selected object. Thus, the processed selected data generated by the target model eliminates the deviation between the processed selected data and the baseline data. It can be understood that this embodiment uses a single adjustment as an example to eliminate the deviation between the processed selected data and the baseline data. However, the present disclosure is not limited thereto. In the case of a large deviation, multiple adjustments may be performed, and the currently generated processed rich text data block may be used as the rich text data block to be processed in the next adjustment. The purpose of each adjustment is to reduce the deviation until the deviation is eliminated or reduced to a negligible level after multiple adjustments.
[0100] It can be understood that the above describes the adjustment method of the rich text data block of the present disclosure, and the following describes some methods of obtaining the target rich text data block.
[0101] In some embodiments, the method may further include: in response to receiving confirmation text for the processed rich text data block, determining the processed rich text data block as a target rich text data block for the image to be processed. For example, the confirmation text may be "This form is now correct." The confirmation text may be user-provided. The target model may determine the processed rich text data block as the target rich text data block.
[0102] It is understood that the above description of the present disclosure uses the example of the adjustment text and confirmation text being input by the user. However, the present disclosure is not limited thereto, and the adjustment text may also be determined by a model, as will be explained below.
[0103] In some embodiments, the above method may further include: determining the adjustment text according to the difference between the rendered image and the image to be processed.
[0104] For example, a large model can be used as a correction large model. The correction large model can determine the difference between the rendered image and the image to be processed and determine the adjustment text. For another example, the target model can also be used to determine the adjustment text. Through the embodiments of the present disclosure, the adjustment text can be determined by the model to further reduce the labor cost of rich text data extraction and improve data generation efficiency. It can be understood that when the adjustment text is determined by the large model, the benchmark data can be obtained by the large model from the image to be processed.
[0105] For another example, if the modified large model determines that the difference between the rendered image used to process the rich text data block and the image to be processed is small or the difference evaluation value is less than a preset difference threshold, a confirmation text can be generated and provided to the target model.
[0106] It is understood that the above description of the present disclosure uses the background color in attribute information as an example. However, the present disclosure is not limited to this. Attribute information can also indicate various attributes such as border style, text font, color, size, alignment, and whether to merge. If one or more of these attributes of the candidate object deviates from the baseline data, these can also be adjusted using the methods of the present disclosure.
[0107] It can be understood that the above describes the reasoning method of the target model of the present disclosure, and the following describes the training method of the target model.
[0108] In some embodiments, the method may further include: determining a sample rich text data block of the sample image using the large model; determining a to-be-processed rich text data block of the sample image using the to-be-trained model; and adjusting parameters of the to-be-trained model based on the differences between the sample rich text data block and the to-be-processed rich text data block to perform knowledge distillation to obtain a target model. The large model may have more parameters than the target model. The large model may serve as a teacher model, and the to-be-trained model may serve as a student model.
[0109] It can be understood that the above describes the text data processing method of the present disclosure, and the following describes the question and answer information processing method of the present disclosure.
[0110] Figure 4 is a flowchart of a question and answer information processing method according to another embodiment of the present disclosure.
[0111] like Figure 4 As shown, the method 400 may include operations S410 to S420.
[0112] In operation S410 , context information for the question information is determined from a knowledge base based on the received question information.
[0113] For example, the question information can be question text or information in any modality. Taking the question text as an example, one or more keywords can be determined from the question text and searched in the knowledge base to obtain context information for the question text. The context information can be context text or information in any modality. It is understood that information in any modality can include at least one of various modalities, such as text, image, audio, and video.
[0114] In operation S420, answer information for the question information is generated according to the context information.
[0115] For example, the large model can generate answer information based on the context information, and the answer information can be displayed on a visual interface to provide it to the user. The answer information can be answer text or information in any modality.
[0116] In an embodiment of the present disclosure, a knowledge base includes multiple target rich text data blocks determined from multiple images to be processed. The target rich text data blocks are determined by the following operations: in response to receiving adjustment text for the rich text data block to be processed, determining at least one descriptive text for at least one selected object in the adjustment text, wherein the rich text data block to be processed is obtained based on the image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjustment text is used to indicate at least one selected object from the at least one candidate object; generating a processed rich text data block for the adjusted text based on the at least one descriptive text for the at least one selected object; and in response to receiving confirmation text for the processed rich text data block, determining the processed rich text data block as the target rich text data block for the image to be processed. It will be understood that the above description of method 200 is also applicable to the generation of the processed rich text data block in this embodiment, and will not be repeated here in this disclosure. In addition, the above description of the target rich text data block is also applicable to this embodiment, and will not be repeated here in this disclosure.
[0117] It can be understood that the above describes the method of the present disclosure, and the following will describe the device of the present disclosure.
[0118] Figure 5 is a block diagram of a text data generating apparatus according to an embodiment of the present disclosure.
[0119] like Figure 5 As shown, the apparatus 500 may include a first determining module 510 and a first generating module 520 .
[0120] A first determining module 510 is configured to, in response to receiving adjusted text for a rich text data block to be processed, determine at least one descriptive text for at least one selected object in the adjusted text. The rich text data block to be processed is obtained based on the image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object from the at least one candidate object.
[0121] The first generating module 520 is configured to generate a processed rich text data block according to at least one description text for at least one selected object.
[0122] In some embodiments, the rich text data block to be processed includes at least one candidate data, the selected object among the at least one candidate object is represented by the selected data among the at least one candidate data, and the descriptive text is used to describe: the deviation between the selected data for the selected object and the benchmark data for the selected object.
[0123] In some embodiments, the processed rich text data block includes at least one processed selected data for at least one selected object, where the processed selected data is used to represent the processed selected object corresponding to the selected object. The first generation module 520 includes a generation submodule for generating the processed selected data for the selected object using a target model based on the descriptive text for the selected object, thereby reducing the deviation between the processed selected data and the baseline data.
[0124] In some embodiments, the candidate data is generated by the target model based on the first reasoning path and the first internal representation data, and the generation submodule includes at least one of the following units: a first generation unit for generating processed selected data based on the second reasoning path; and a second generation unit for generating processed selected data based on the second internal representation data.
[0125] In some embodiments, the apparatus 500 further includes: a second determining module configured to determine the processed rich text data block as a target rich text data block of the image to be processed in response to receiving a confirmation text for the processed rich text data block.
[0126] In some embodiments, the rich text data block to be processed includes at least one candidate data for at least one candidate object, the candidate data including attribute information for the candidate object, the attribute information being used to indicate at least a visual style of the candidate object. Apparatus 500 further includes a rendering module configured to render the rich text data block to be processed to generate a rendered image.
[0127] In some embodiments, the apparatus 500 further includes at least one of the following modules: a third determining module configured to determine the adjustment text according to the difference between the rendered image and the image to be processed; and an acquiring module configured to acquire the adjustment text input by the user.
[0128] In some embodiments, the image to be processed is a table image to be processed, and the at least one candidate object includes a candidate table object, at least one candidate row object, and at least one candidate cell object.
[0129] Figure 6 is a block diagram of a question and answer information processing apparatus according to another embodiment of the present disclosure.
[0130] like Figure 6 As shown, the apparatus 600 may include a fourth determining module 610 and a second generating module 620 .
[0131] The fourth determining module 610 is configured to determine context information for the question information from a knowledge base according to the received question information.
[0132] The second generating module 620 is configured to generate answer information for the question information according to the context information.
[0133] In an embodiment of the present disclosure, the knowledge base includes multiple target rich text data blocks determined from multiple images to be processed, and the target rich text data blocks are determined by the following modules: a first determination module, which is used to determine at least one descriptive text for at least one selected object in the adjusted text in response to receiving an adjustment text for the rich text data block to be processed, wherein the rich text data block to be processed is obtained based on the image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object among the at least one candidate object. A first generation module, which is used to generate a processed rich text data block for the adjusted text based on the at least one descriptive text for the at least one selected object. A second determination module, which is used to determine the processed rich text data block as the target rich text data block of the image to be processed in response to receiving a confirmation text for the processed rich text data block. For example, the target rich text data block can be determined by the above-mentioned device 500.
[0134] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0135] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0136] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0137] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. Computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.
[0138] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0139] The computing unit 701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as at least one of the text data processing method and the question-and-answer information processing method. For example, in some embodiments, at least one of the text data processing method and the question-and-answer information processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of at least one of the text data processing method and the question-and-answer information processing method described above can be performed. Alternatively, in other embodiments, computing unit 701 can be configured to perform at least one of the text data processing method and the question-and-answer information processing method using any other suitable means (e.g., via firmware).
[0140] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0141] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0142] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0144] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.
[0145] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.
[0146] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.
[0147] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.
Claims
1. A text data processing method, comprising: In response to receiving adjusted text for a rich text data block to be processed, determining at least one descriptive text for at least one selected object in the adjusted text, wherein the rich text data block to be processed is obtained based on an image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object from the at least one candidate object; A processed rich text data block for the adjusted text is generated based on at least one of the description texts for at least one of the selected objects.
2. The method according to claim 1, wherein The rich text data block to be processed includes at least one candidate data, and the selected object in at least one candidate data is represented by the selected data in at least one candidate data. The description text is used to describe: a deviation between the selected data for the selected object and the reference data for the selected object.
3. The method according to claim 2, wherein: The processed rich text data block includes at least one processed selected data for at least one selected object, wherein the processed selected data is used to represent the processed selected object corresponding to the selected object. Generating a processed rich text data block for the adjusted text based on at least one description text for at least one selected object includes: Processed selected data for the selected object is generated using a target model according to the description text for the selected object, so as to reduce a deviation between the processed selected data and the reference data.
4. The method according to claim 3, wherein: The candidate data is generated by the target model based on the first reasoning path and the first internal representation data, Generating processed selected data for the selected object using the target model comprises at least one of the following operations: generating the processed selected data based on the second reasoning path; The processed selected data is generated based on the second internal representation data.
5. The method according to claim 1, further comprising: In response to receiving the confirmation text for the processed rich text data block, the processed rich text data block is determined as the target rich text data block of the image to be processed.
6. The method according to claim 1, wherein The to-be-processed rich text data block includes at least one candidate data for at least one candidate object, the candidate data includes attribute information for the candidate object, and the attribute information is at least used to indicate a visual style of the candidate object. Also includes: The rich text data block to be processed is rendered to obtain a rendered image.
7. The method according to claim 6, further comprising at least one of the following operations: determining the adjustment text according to a difference between the rendered image and the image to be processed; Get the adjustment text entered by the user.
8. The method according to any one of claims 1 to 7, wherein: The image to be processed is a table image to be processed, and the at least one candidate object includes a candidate table object, at least one candidate row object, and at least one candidate cell object.
9. A method for processing question-answer information, comprising: Determining context information for the question information from a knowledge base based on the received question information; generating answer information for the question information based on the context information, The knowledge base includes a plurality of target rich text data blocks determined from a plurality of images to be processed, and the target rich text data blocks are determined by the following operations: In response to receiving adjusted text for a rich text data block to be processed, determining at least one descriptive text for at least one selected object in the adjusted text, wherein the rich text data block to be processed is obtained based on an image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object from the at least one candidate object; generating a processed rich text data block for the adjusted text based on at least one of the description texts for at least one of the selected objects; In response to receiving the confirmation text for the processed rich text data block, the processed rich text data block is determined as the target rich text data block of the image to be processed.
10. A text data processing device, comprising: a first determining module configured to, in response to receiving adjusted text for a rich text data block to be processed, determine at least one descriptive text for at least one selected object in the adjusted text, wherein the rich text data block to be processed is obtained based on an image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object from the at least one candidate object; The first generating module is configured to generate a processed rich text data block for the adjusted text based on at least one description text for at least one selected object.
11. A question-answer information processing device, comprising: a fourth determining module, configured to determine context information for the question information from a knowledge base based on the received question information; A second generating module is configured to generate answer information for the question information based on the context information, The knowledge base includes a plurality of target rich text data blocks determined from a plurality of images to be processed, and the target rich text data blocks are determined by the following modules: a first determining module configured to, in response to receiving adjusted text for a rich text data block to be processed, determine at least one descriptive text for at least one selected object in the adjusted text, wherein the rich text data block to be processed is obtained based on an image to be processed, the rich text data block to be processed is used to represent at least one candidate object, and the adjusted text is used to indicate at least one selected object from the at least one candidate object; A first generating module, configured to generate a processed rich text data block for the adjusted text based on at least one description text for at least one selected object; The second determining module is configured to, in response to receiving a confirmation text for the processed rich text data block, determine the processed rich text data block as a target rich text data block of the image to be processed.
12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.
13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.
14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.