Data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a data processing method, relates to the technical field of artificial intelligence, and particularly relates to the technical field of large models, deep learning, natural language processing and computer vision. A specific implementation scheme is as follows: in response to receiving a query text for input data, obtaining a data structure of the input data, wherein the data structure comprises nodes of multiple levels, each node containing content of the input data; obtaining target text contained in a node with text content from the data structure; and determining output data based on the query text and the target text. The present disclosure also provides a data processing device, an electronic device and a storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the fields of large models, deep learning, natural language processing, and computer vision. More specifically, this disclosure provides a data processing method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] With the rapid development of information technology, users' needs for document resources in fields such as office work and education are becoming increasingly diversified. Especially in terms of quickly extracting and integrating information from multiple sources, traditional document retrieval and reading methods can no longer meet users' needs for quickly obtaining the core content of documents and achieving personalized information integration. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, electronic device, storage medium, and computer program product.

[0004] According to a first aspect, a data processing method is provided, the method comprising: in response to receiving a query text for input data, obtaining a data structure of the input data, wherein the data structure includes multiple levels of nodes, each node containing the content of the input data; obtaining target text contained in nodes whose content is text from the data structure; and determining output data based on the query text and the target text.

[0005] According to a second aspect, a data processing apparatus is provided, comprising: a data structure acquisition module, configured to acquire a data structure of the input data in response to receiving a query text for the input data, wherein the data structure includes multiple levels of nodes, each node containing the content of the input data; a target text determination module, configured to acquire the target text contained in the nodes whose content is text from the data structure; and an output data determination module, configured to determine output data based on the query text and the target text.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to the present disclosure.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the methods provided in this disclosure.

[0008] According to a fifth aspect, a computer program product is provided, comprising a computer program stored on at least one of a readable storage medium and an electronic device, wherein the computer program, when executed by a processor, implements the method provided in this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 This is an exemplary system architecture diagram of a data processing method and apparatus applicable according to an embodiment of the present disclosure;

[0012] Figure 2 This is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0013] Figure 3 This is a flowchart of a data processing method according to another embodiment of the present disclosure;

[0014] Figure 4 This is a schematic diagram of a data processing system according to an embodiment of the present disclosure;

[0015] Figure 5 This is a flowchart of a data processing method according to another embodiment of the present disclosure;

[0016] Figure 6 This is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0017] Figure 7 This is a block diagram of an electronic device for a data processing method according to an embodiment of the present disclosure. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] Traditional document retrieval systems mainly rely on keyword search and manual reading, which have limited ability to integrate and summarize multi-source data, making it difficult to meet users' expectations for quickly and accurately obtaining the core content of documents.

[0020] Multi-source data refers to data from different sources and of different types, such as documents, images, and web pages. Currently, different processing methods are often used for multi-source data. For example, document processing typically relies on text editing software such as Microsoft Word and Adobe PDF. Image processing usually depends on image recognition technologies in the field of computer vision, such as extracting image information through OCR (Optical Character Recognition) technology. Web page data is often obtained by crawling web pages using web scraping techniques.

[0021] Despite significant advancements in related technologies for document processing, web scraping, and image recognition, some obvious shortcomings remain when handling multi-source data. These include the following aspects.

[0022] Limitations of Single Technologies: Currently, these document processing, web scraping, and image recognition technologies are developed independently, lacking unified integration and coordination. This leads to the need to use different tools or technologies when processing multi-source data, resulting in inefficiency and a high risk of errors.

[0023] Lack of intelligence and automation: Currently, processing multi-source data often requires human intervention and lacks intelligent and automated capabilities. For example, document processing requires manual reading and extraction of key information; web scraping requires manually writing crawler programs and dealing with anti-scraping mechanisms; and image recognition and understanding requires manual annotation of training data. These manual operations are not only time-consuming and labor-intensive, but also prone to introducing subjectivity and errors.

[0024] Low processing efficiency: When processing multi-source data, there is a lack of efficient data processing algorithms and models, making it impossible to quickly and accurately extract and analyze key information from the data.

[0025] Limited cross-format processing capabilities: When processing multi-source data, there are often limited supports for different types of files or data formats. For example, some document processing tools may only support document files of specific formats; some web crawlers may not be able to crawl data from certain websites; some image recognition systems may not be able to recognize certain types of images, etc. This limits the breadth and applicability of these technologies in practical applications.

[0026] In view of this, embodiments of the present disclosure provide a data processing method that, in response to receiving a query text for input data, obtains a data structure of the input data, wherein the data structure includes multiple levels of nodes, each node containing the content of the input data; obtains the target text contained in the nodes whose content is text from the data structure; and determines output data based on the query text and the target text. The data processing method provided by embodiments of the present disclosure can automate and intelligently process files, improving processing efficiency and accuracy. When the input data is multi-source data, it can integrate the multi-source data, summarize and answer questions based on the multi-source data, enabling more efficient processing of multi-source data and generating more accurate and valuable summary and question-and-answer results.

[0027] The data processing methods and apparatus provided in this disclosure will be described in detail below.

[0028] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0029] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0030] Figure 1 This is a schematic diagram of an exemplary system architecture to which data processing methods and apparatus can be applied according to an embodiment of this disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices, including but not limited to smartphones, tablets, laptops, etc.

[0033] The data processing method provided in this disclosure can generally be executed by server 105. Correspondingly, the data processing apparatus provided in this disclosure can generally be located in server 105. The data processing method provided in this disclosure can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the data processing apparatus provided in this disclosure can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0034] Figure 2 This is a flowchart of a data processing method according to an embodiment of the present disclosure.

[0035] like Figure 2 As shown, the data processing method 200 includes operations S210 to S230.

[0036] In operation S210, in response to receiving a query text for the input data, the data structure of the input data is obtained.

[0037] Input data can be user-uploaded files, such as at least one of the following: documents, images, and web pages. Documents can include Word, PDF, PPT, Excel, etc. Images can include regular images as well as photocopies, such as scanned PDFs.

[0038] The query text can be a question entered by the user in response to the uploaded input data. For example, the user might upload multiple files that are the same or different types of files. The query text could include phrases like "summarize the key points of multiple files," "select the better file from file A and file B and output it," "what is the specific meaning of the XXX issue mentioned in file A," or "regenerate a document based on file B," etc.

[0039] The input data structure can be obtained by parsing the received input data, and the resulting data structure can be stored. Alternatively, the input data structure can be obtained from stored historical parsing results. The data structure consists of multiple nodes with a hierarchical relationship, and each node contains relevant content about the input data.

[0040] For example, these nodes can include page nodes, heading nodes, paragraph nodes, etc. The content of a heading node is the heading content. The content of a paragraph node is the paragraph content; paragraph nodes can also include text nodes, image nodes, table nodes, formula nodes, object nodes, etc. The content of a text node includes text, the content of an image node includes an image, and the content of a table node includes the content of cells within a table. Paragraph nodes can also include nodes for styles, attributes, and positions.

[0041] After receiving the query text for the input data, the data structure of the input data can be retrieved from the stored data structure. When the input data contains multiple files, the data structure of the specified file can be retrieved based on the file specified in the query text.

[0042] In operation S220, the target text contained in the node whose content is text is obtained from the data structure.

[0043] For example, you can get the title text content from the title node and the paragraph text content from the paragraph node.

[0044] A title node can include text nodes and style nodes. Text nodes store the title text content, while style nodes store the title's style, such as font, size, and position. The title text content can be retrieved from the text node.

[0045] Paragraph nodes can include text nodes, image nodes, table nodes, formula nodes, object nodes, and style nodes. The paragraph text content can be retrieved from the text node.

[0046] The text content obtained from the above text nodes can be used as the target text.

[0047] In operation S230, the output data is determined based on the query text and the target text.

[0048] It can be combined with large model technology to generate output data based on query text and target text.

[0049] For example, query text and target text can be input into a large model, which can then summarize, answer questions, or create content based on the query text, generating answers to the query text, creating new content, etc., and returning it to the user.

[0050] When the input data is multi-source data, such as multiple files of different types, a large model can be used to generate output data based on the multi-source data.

[0051] For example, the query text and the target text of multiple files can be input into a large model. The large model can then summarize, refine, answer questions, and create content based on the multi-source data, generating answers to the query text, creating new content, and returning it to the user.

[0052] The embodiments of this disclosure obtain the data structure of the input data, extract the target text from the data structure, and generate output data based on the query text and target text using a large model. This enables automated and intelligent file processing, improving processing efficiency and accuracy.

[0053] When input data comes from multiple sources, the unified processing of these data into a standardized data structure enables cross-format processing, allowing for more efficient utilization of data from different sources. Furthermore, leveraging large models for summarizing, answering questions, and generating content from multi-source data allows for more efficient processing and produces more accurate and valuable summaries and answers.

[0054] Figure 3 This is a flowchart of a data processing method according to another embodiment of the present disclosure.

[0055] like Figure 3 As shown, the data processing method includes operations S310 to S360.

[0056] The S310 is in operation, receiving input data.

[0057] Input data can be files uploaded by users, and the files can be of different types, such as documents, images, web pages, etc.

[0058] In operation S320, determine if the input data has historical parsing results. If so, execute operation S340; otherwise, execute operation S330.

[0059] For user-uploaded files, we can first determine if the file has historical parsing results. Historical parsing results include the data structure of the file that has already been parsed. In one example, we can pre-set a document library containing the parsing results (i.e., data structures) of multiple pre-set files. These pre-set files can be frequently used by the user or files with high application value. We can also pre-set a historical parsing database to store the data structures of files that the user has uploaded and parsed. The data structures in both the document library and the historical parsing results database are historical parsing results.

[0060] It can determine whether the preset document library and database contain the data structure of the file currently uploaded by the user. If yes, then operation S340 is executed. If no, then operation S330 is executed.

[0061] The S330 is used to parse the input data, obtain the data structure of the input data, and store it.

[0062] According to embodiments of this disclosure, for each type of file, multiple hierarchical structures and the content of each hierarchical structure are parsed from the file; and the input data is converted into a data structure with each level as a node and the content of the level as the content of the node.

[0063] For example, for documents such as Word, PPT, and Excel, which are editable text, direct conversion (encoding) can be performed. For instance, a hierarchical structure can be determined based on the document structure, and the content of each level can be determined based on the document content. The levels are then used as nodes, and the content of each level is used as the content of each node, resulting in a data structure.

[0064] For images and photocopied PDFs, text can be extracted from the images. Image understanding technology can also be used to generate image-understood content, which is then integrated with the image recognition content to form a complete descriptive text for the image. This complete descriptive text is then transcoded to obtain the data structure.

[0065] For web pages, web crawling technology can be used to obtain information such as the website, topic, and content, forming text. This text is then transcoded to obtain the web page's data structure.

[0066] The aforementioned documents, images, and web pages are all converted into a unified data structure, which can be a bdjson data structure.

[0067] In operation S340, in response to the query text, the data structure of the input data is obtained from the current parsing result / historical parsing result.

[0068] The current parsing result is the data structure of the file obtained from operation S330. Historical parsing results are the data structures of files stored in the document library or historical parsing database.

[0069] After receiving a user's question, the data structure of the file the user wants to query can be obtained from the current parsing results or historical parsing results.

[0070] In operation S350, the target text is extracted from the acquired data structure.

[0071] For example, the title text can be retrieved from the text nodes under the title node, and the paragraph text content can be retrieved from the text nodes under the paragraph node. The text content retrieved from each of these text nodes will be used as the target text.

[0072] When operating S360, output data is generated based on the query text and target text.

[0073] According to embodiments of this disclosure, a large model is used to summarize, answer questions, and create at least one of the following based on the query text to obtain output data.

[0074] For example, query text and target text can be input into a large model, which can then perform at least one of the following actions based on the query text: summarizing, question answering, or creation, to generate an answer to the query text and return it to the user.

[0075] According to embodiments of this disclosure, when the input data is multi-source data, such as the input data including multiple files of different types, the target text of each of the multiple files is integrated based on the query text to obtain integrated text. At least one of summarizing and answering questions is then performed on the integrated text to obtain output data.

[0076] For example, the query text and the target text of each of the multiple files can be input into a large model. The large model integrates the multiple target files to obtain integrated text. Based on the integrated text, it performs at least one of the following: summarizing, refining, question answering, and creation, to generate the answer to the query text and return it to the user.

[0077] The embodiments of this disclosure, when dealing with user-uploaded files, directly retrieve the data structure from the historical parsing results if the file's data structure is included in the historical parsing results. This avoids the overhead of repeated parsing and transcoding, thereby improving system response speed and user experience.

[0078] The embodiments of this disclosure, when the input data is multi-source data, can summarize, answer questions and create content based on the multi-source data, process multi-source data more efficiently, and generate more accurate and valuable summary and question-and-answer results.

[0079] Figure 4 This is a schematic diagram of a data processing system according to an embodiment of the present disclosure.

[0080] like Figure 4 As shown, the data processing system includes a user service module 410, a multi-source data processing module 420, and a large model service module 430.

[0081] User service module 410 is a platform that provides large model services to users. On this platform, users can upload various file types, such as WORD, PPT, ESCEL, photocopied PDFs, images, and web URLs (Uniform Resource Locator). Users can also choose from summary scenarios, question-and-answer scenarios, and creation scenarios, and can input user questions within each scenario.

[0082] The multi-source data processing module 420 includes a parsing buffer 421, a parsing submodule 422, and a storage submodule 423.

[0083] The parsing submodule 422 includes a document parser, a photocopy parser, an image parser, and a webpage parser. The document parser, for documents such as Word, PPT, and Excel, determines the multiple hierarchical structures of the document and the content of each level based on the document structure and content, and then transcodes it to obtain the document's data structure. The webpage parser, for webpage files, determines the multiple hierarchical structures of the webpage and the content of each level based on the webpage structure and page content, and then transcodes it to obtain the webpage's data structure. The photocopy parser, for photocopied images, determines the multiple hierarchical structures of the image and the content of each level based on image understanding and image recognition, and then transcodes it to obtain the photocopy image's data structure. The image parser functions similarly to the photocopy parser, determining the multiple hierarchical structures of the image and the content of each level based on image understanding and image recognition, and then transcodes it to obtain the image's data structure. The photocopy decoder can focus on parsing photocopied images (such as scanned PDFs).

[0084] Each file type is converted into a unified data structure by its respective parser. This unified data structure can be a bdjson format data structure.

[0085] According to embodiments of this disclosure, an image understanding text of an image is generated; an image recognition text of the image is obtained; an overall description text of the image is generated based on the image understanding text and the image recognition text; and multiple hierarchical structures and the content of each hierarchical structure are parsed from the overall description text.

[0086] For example, an image understanding model can be used to generate image-understood text, and an OCR recognition model can be used to obtain image-recognized text. The image-understood text and image-recognized text are then integrated to obtain the overall image description text. Next, the hierarchical structure of the overall image description text is parsed, and the hierarchical structure is converted into a data structure.

[0087] The parsing submodule 422 in this embodiment can not only support the parsing of different types of files, but also the simultaneous parsing of a large number of files. For example, it can support the simultaneous parsing of 100 files of different types, greatly improving the efficiency of file parsing and transcoding.

[0088] According to embodiments of this disclosure, for each file, a mapping relationship between the file and its data structure is determined; and the data structures and mapping relationships of multiple files are stored respectively.

[0089] The storage submodule 423 is used to store the parsed data structures. Storage submodule 423 may include a preset document library for storing preset text data structures. It may also include a historical parsing database for storing the data structures of files uploaded and parsed by the user. The data structures in both the preset document library and the historical parsing database are historical parsing results.

[0090] Parsing buffer 421 is used to store the mapping relationship between files and the parsing results of files. For example, it maps file IDs to the storage addresses of file data structures and writes the mapping relationship into parsing buffer 421.

[0091] This embodiment maps files and their parsing results to distinguish the parsing results for different types of files. By writing the mapping relationship into the parsing cache 421, previously parsed data can be directly utilized by reading the parsing cache, avoiding the overhead of repeated parsing and transcoding. Furthermore, it achieves fast request response and efficient result output.

[0092] The large model service module 430 provides large model services such as document summarization, document Q&A, and document creation. For example, a user inputs a question and requests a large model service. Based on the target text of the document, the large model can summarize the key points of the document, extract the core content, etc., and return it to the user. The large model receives the user's query text, and based on the query text and the target text of the document, generates answers to the query text, creates new content, etc., and returns it to the user.

[0093] For example, large models can output the generated results to the client using the SSE (Server-Sent Event) protocol. The SSE protocol is a streaming output method, which can continuously output long results in segments, rather than waiting for all results to be generated before returning the long result all at once. This can avoid long waiting times for users and improve the user experience.

[0094] The data processing system of this disclosure is applicable to a wide range of office, education, and research fields. For example, in an office setting, users can use the system to quickly integrate multiple documents, images, and web pages, extract key content, and provide support for decision-making. In education, teachers can use the system to quickly organize teaching materials, generate teaching summaries and answers to questions, and improve teaching efficiency. In research, researchers can use the system to quickly integrate and analyze large amounts of literature, providing strong support for scientific research. Furthermore, the system can also be applied to multiple fields such as law, finance, and medicine, helping users quickly obtain and analyze relevant information, improving work efficiency and decision-making quality.

[0095] Compared to traditional document retrieval and processing systems, the data processing system provided in this disclosure has the following advantages.

[0096] The data processing system disclosed herein enables integrated processing of multi-source data. The system can simultaneously process different types of data sources, such as documents, images, and web page URLs, achieving a unified processing workflow for multi-source data. This integrated processing approach significantly improves the efficiency and accuracy of data processing.

[0097] The data processing system disclosed herein implements intelligent parsing and transcoding. Different intelligent parsing and transcoding technologies are employed for different types of files. For example, for webpage URLs, the system uses web crawling technology to extract information and adapt it into a unified format; for images, the system utilizes AI image recognition and OCR technologies to extract image content; and for documents, the system generates a unified format result through a real-time transcoding service. This intelligent parsing and transcoding technology enables the system to more accurately understand and process various types of data.

[0098] The data processing system disclosed herein provides a unified data format. The system adapts and transforms parsed data into a unified data structure, which facilitates subsequent data processing and model applications. By adopting a unified data format, the system can more efficiently utilize data from different sources and improves the consistency of data processing and its scalability to support more data types.

[0099] The data processing system disclosed herein incorporates large model application technology. After data parsing and transcoding, the system uses large model application technology to summarize and answer questions about the data. This approach, combining large model application technology with data analysis, enables the system to gain a deeper understanding and analysis of the data, and generate more accurate and valuable summaries and answers.

[0100] The data processing system disclosed herein achieves highly efficient caching and request processing. By reading the parsing cache, the system can directly utilize historically parsed data, avoiding the overhead of repeated parsing and transcoding. Simultaneously, the system writes the mapping relationship between file IDs and parsing results into the parsing cache, enabling fast request response and efficient result output. This efficient caching and request processing method improves the system's response speed and user experience.

[0101] Figure 5 This is a flowchart of a data processing method according to another embodiment of the present disclosure.

[0102] like Figure 5 As shown, this embodiment includes operations S510 to S590.

[0103] When operating the S510, it receives files uploaded by the user.

[0104] When operating the S520, determine whether a file has passed the filter.

[0105] For example, a file is input into a filter, which determines whether the file contains sensitive or illegal information. If the filter determines that the file contains sensitive or illegal information, operation S590 is executed, and data is output. This output data indicates to the user that the uploaded file contains sensitive or illegal information.

[0106] If the file passes the filter's judgment, meaning it does not contain sensitive or illegal information, then operation S530 can be executed.

[0107] When operating the S530, determine if the file has historical parsing results.

[0108] According to embodiments of this disclosure, a data structure for obtaining input data from historical parsing results is derived based on a mapping relationship.

[0109] For example, the parsing buffer can be read to determine if it stores a mapping relationship between files and data structures. If so, it means that the file has historical parsing results, and the previously parsed data structures can be directly used to execute operation S580, avoiding duplicate parsing.

[0110] If the parsing buffer does not store the mapping between files and data structures, operation S540 can be performed.

[0111] When operating the S540, determine the file type.

[0112] When operating the S550, files of different types are processed by their respective parsers and converted into their respective data structures.

[0113] The S560 is used to store the data structure of each parsed file.

[0114] File types can include documents such as Word, PPT, and Excel, photocopied PDFs, images, and web pages. Different parsers are selected for different file types to process them, resulting in the respective data structures for each file type, which are then stored.

[0115] During operation of S570, the mapping relationship between files and their respective data structures is written to the parsing buffer.

[0116] For example, the file ID is mapped to the storage address of the file's data structure, and the mapping is written to the parsing cache.

[0117] When operating the S580, in response to user requests, the large model is invoked to perform services such as summarizing, answering questions, and creating content.

[0118] For example, a user can request services such as document summary, content selection, document Q&A, and content creation. In response to a user request, the large model can perform at least one of these services: summarizing, comparing, Q&A, and content creation.

[0119] When operating the S590, output data.

[0120] For example, outputting the key points summarized by the large model, the generated answers, and the new content created.

[0121] The following section uses specific scenarios and examples to explain the capabilities of this publicly available large model, including its summarization, Q&A, and creation functions.

[0122] Table 1 below is an example of a summary scenario.

[0123] Table 1

[0124]

[0125] Table 2 below is an example of a content optimization scenario.

[0126] Table 2

[0127]

[0128] Table 3 below shows an example of a question-and-answer scenario.

[0129] Table 3

[0130]

[0131] Table 4 below is an example of a creative scenario.

[0132] Table 4

[0133]

[0134] The embodiments disclosed herein, for multi-source data, combine large models to summarize, answer questions, and create content based on multi-source data, which can process multi-source data more efficiently and generate more accurate and valuable summary and question-and-answer results.

[0135] According to embodiments of this disclosure, this disclosure also provides a data processing apparatus.

[0136] Figure 6 This is a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0137] like Figure 6 As shown, the data processing device 600 includes a data structure acquisition module 610, a target text determination module 620, and an output data determination module 630.

[0138] The data structure acquisition module 610 is used to acquire the data structure of the input data in response to receiving a query text for the input data. The data structure includes nodes at multiple levels, and each node contains the content of the input data.

[0139] The target text determination module 620 is used to obtain the target text contained in the nodes whose content is text from the data structure.

[0140] The output data determination module 630 is used to determine the output data based on the query text and the target text.

[0141] The data processing device 600 also includes a receiving module and a parsing module.

[0142] The receiving module is used to receive input data.

[0143] The parsing module is used to parse the input data to obtain the data structure.

[0144] The input data includes multiple files of different types. The parsing module includes parsing units and conversion units.

[0145] The parsing unit is used to parse multiple hierarchical structures and the content of each hierarchical structure from a file of each type.

[0146] The transformation unit is used to convert input data into a data structure, with each level as a node and the content of the level as the content of the node.

[0147] The multiple files include at least one of documents, web pages, and images. The parsing unit includes a document parsing subunit, a web page parsing subunit, and an image parsing subunit.

[0148] The document parsing subunit is used to determine the multiple hierarchical structures of a document and the content of each hierarchical structure based on the document structure and document content.

[0149] The webpage parsing subunit is used to determine the multiple hierarchical structures of a webpage and the content of each hierarchical structure based on the page structure and page content.

[0150] The image parsing subunit is used to determine the multiple hierarchical structures of an image and the content of each hierarchical structure based on the image understanding content and image recognition content.

[0151] The image parsing subunit is used to generate image understanding text for an image; obtain image recognition text for an image; generate an overall description text for the image based on the image understanding text and the image recognition text; and parse multiple hierarchical structures and the content of each hierarchical structure from the overall description text.

[0152] The data processing device 600 also includes a mapping relationship determination module and a storage module.

[0153] The mapping relationship determination module is used to determine the mapping relationship between each file and its data structure.

[0154] The storage module is used to store the data structures and mapping relationships of multiple files.

[0155] The data structure acquisition module 610 is used to obtain the data structure of the input data from the historical parsing results according to the mapping relationship.

[0156] The output data determination module 630 is used to obtain output data by summarizing, answering questions, and creating at least one of the following methods based on the query text and the target text.

[0157] The output data determination module 630 is also used to integrate the target texts of multiple files based on the query text to obtain integrated text; and to perform at least one of the following on the integrated text: summarizing, question answering, and creation to obtain output data.

[0158] The nodes include heading nodes and paragraph nodes. Paragraph nodes include at least one of the following: text nodes, image nodes, table nodes, formula nodes, and style nodes.

[0159] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0160] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0161] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0162] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0163] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0164] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0165] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0166] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0167] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0168] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0169] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0170] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0171] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method, comprising: Receive input data, which includes multiple files of different types; Each file is parsed to obtain the file's data structure, which includes multiple levels of nodes. Each node contains the content of the corresponding level of the file. The parsing includes: for image files, generating image understanding text using an image understanding model, recognizing image recognition text from the image using a text recognition model, generating an overall description text of the image based on the image understanding text and the image recognition text, and parsing multiple hierarchical structures and the content of each hierarchical structure from the overall description text. For each file, determine the mapping relationship between the file and its data structure; Store the data structure and mapping relationship of each of the multiple files; In response to receiving a query text for the input data, determine the file specified by the query text, and obtain the data structure of the specified file according to the mapping relationship; Obtain the target text contained in nodes whose content is text from the data structure; and Based on the query text and the target text, determine the output data.

2. The method according to claim 1, wherein, The process of parsing each file to obtain its data structure includes: for each file type, Parse multiple hierarchical structures and the content of each hierarchical structure from the file; and The input data is converted into a data structure by using each level as a node and the content of the level as the content of the node.

3. The method according to claim 2, wherein, The plurality of files also includes at least one of document files and web page files; the parsing of the plurality of hierarchical structures and the content of each hierarchical structure from the files includes at least one of the following operations: Based on the document structure and document content, determine the multiple hierarchical structures of the document and the content of each hierarchical structure; Based on the page structure and page content, determine the multiple hierarchical structures of the webpage and the content of each hierarchical structure.

4. The method according to claim 1, wherein, The process of determining the output data based on the query text and the target text includes: Based on the query text, the target text is summarized, question-and-answered, or created in at least one of these ways to obtain the output data.

5. The method according to claim 1, wherein, The input data includes multiple files of different types; the step of determining the output data based on the query text and the target text includes: Based on the query text, the target texts of the multiple files are integrated to obtain the integrated text; The integrated text is summarized, question-and-answered, or created in at least one of these ways to obtain the output data.

6. The method according to any one of claims 1 to 5, wherein, The nodes include title nodes and paragraph nodes, and the paragraph nodes include at least one of text nodes, image nodes, table nodes, formula nodes, and style nodes.

7. A data processing apparatus, comprising: A receiving module is used to receive input data, which includes multiple files of different types; The parsing module is used to parse each file to obtain the data structure of the file. The data structure includes multiple levels of nodes, and each node contains the content of the corresponding level of the file. The parsing includes: for image files, generating image understanding text of the image using an image understanding model, recognizing image recognition text from the image using a text recognition model, generating overall description text of the image based on the image understanding text and the image recognition text, and parsing multiple hierarchical structures and the content of each hierarchical structure from the overall description text. A data structure acquisition module is used to acquire the data structure of the input data in response to receiving a query text for the input data, wherein the data structure includes multiple levels of nodes, and each node contains the content of the input data; The target text determination module is used to obtain the target text contained in nodes whose content is text from the data structure; and The output data determination module is used to determine the output data based on the query text and the target text.

8. The apparatus according to claim 7, wherein, The parsing module includes: The parsing unit is used to parse multiple hierarchical structures and the content of each hierarchical structure from a file of each type; and The conversion unit is used to convert the input data into a data structure, with each level as a node and the content of the level as the content of the node.

9. The apparatus according to claim 8, wherein, The plurality of files further includes at least one of document files and web page files; the parsing unit includes: The document parsing subunit is used to determine the multiple hierarchical structures of the document and the content of each hierarchical structure based on the document structure and document content. A webpage parsing subunit is used to determine the multiple hierarchical structures of the webpage and the content of each hierarchical structure based on the page structure and page content; and The image parsing subunit is used to determine the multiple hierarchical structures of the image and the content of each hierarchical structure based on the image understanding content and the image recognition content.

10. The apparatus according to claim 7, wherein, The output data determination module is used to perform at least one of the following on the target text based on the query text: summarizing, question answering, and creation, to obtain the output data.

11. The apparatus according to claim 7, wherein, The input data includes multiple files of different types; the output data determination module is used to integrate the target text of each of the multiple files based on the query text to obtain integrated text; The integrated text is summarized, question-and-answered, or created in at least one of these ways to obtain the output data.

12. The apparatus according to any one of claims 7 to 11, wherein, The nodes include title nodes and paragraph nodes, and the paragraph nodes include at least one of text nodes, image nodes, table nodes, formula nodes, and style nodes.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program implementing the method according to any one of claims 1 to 6 when executed by a processor.

Images