Method and device for dynamically capturing and displaying multi-source document data and medium

Abstract

The invention discloses a method and equipment for dynamically capturing and displaying multi-source data and a medium, and relates to the technical field of receipt sharing. The method comprises the steps that multiple systems are in butt joint, multi-source document data are captured in real time, and the document data comprise tables and images; performing text recognition on the document data through natural language processing to obtain document types and document associated fields; obtaining an upstream field and a downstream field of the receipt name through a regular expression according to the receipt associated field; obtaining a first-level node according to the receipt name, and configuring an ID for the first-level node; searching the document data of which the upstream field is a first-level node to obtain a second-level node; the map is generated according to the nodes, the first-level nodes take the center position of the canvas as the node position, and the second-level nodes are evenly distributed around the first-level nodes by 360 degrees. According to the method, data sharing is realized, and errors and redundancy caused by data islands are reduced; through centralized display and visualization of the data, association and interaction between the data can be visually displayed.

Description

Technical Field

[0001] The present application relates to the technical field of document sharing, and in particular to a method, device and medium for dynamically capturing and displaying multi-source data. Background Art

[0002] With the continuous development of project informatization, the actual progress of the project will generate more and more project data. The data generated in various links of the project are interrelated and interdependent. These data are distributed among different systems, departments and personnel, forming a complex data network. Tracking and tracing these data has become particularly difficult because it is necessary to cross multiple systems and processes to trace the original source of the data, modification history and related decision-making basis.

[0003] The lack of unified data standards and interfaces between systems has resulted in the inability to freely circulate and share data, forming information islands. This not only requires the same data to be repeatedly entered into multiple systems, increasing the workload, but may also lead to data inconsistencies and errors. It is difficult to intuitively see the connections and interactions between different data points, which limits the in-depth understanding of the hidden patterns and associations behind the data.

[0004] Through the above analysis, the problems and defects of the prior art are as follows:

[0005] The data between multiple systems in the existing technology are interrelated, but cannot be shared, displayed and tracked, which limits the connection between the data and makes it impossible to intuitively see the interaction between the data. Summary of the invention

[0006] The embodiments of the present application provide a method, device and medium for dynamically capturing and displaying multi-source data, which can solve the problem in the prior art that data between multiple systems are interrelated but cannot be shared, displayed and tracked, which limits the association between data and cannot intuitively see the interaction between data.

[0007] In a first aspect, an embodiment of the present application provides a method for dynamically capturing and displaying multi-source data, the method comprising: connecting to multiple systems to capture multi-source document data in real time, the document data comprising tables and images; performing text recognition on the document data through natural language processing to obtain the document type and document-related fields; according to the document-related fields, obtaining the upstream field and downstream field of the document name through regular expressions; obtaining a first-level node according to the document name, and configuring an ID for the first-level node; searching for document data whose upstream field is a first-level node to obtain a second-level node; generating a graph according to the nodes, with the first-level node taking the center position of the canvas as the node position, and the second-level nodes being evenly distributed 360° around the first-level node.

[0008] In one implementation of the present application, multiple systems are connected to capture document data from multiple sources in real time, specifically including: polling and monitoring multiple systems, the systems include internal enterprise systems and external enterprise systems; when new document data is captured, the new document data is extracted.

[0009] In one implementation of the present application, text recognition is performed on document data through natural language processing to obtain document types and document-related fields, specifically including: preprocessing new document data to extract text from tables and images; identifying document types and document-related fields through natural language processing, where the document type includes the document name and data source; determining whether the document type and document-related fields exist in the database; if not, saving the document type and document-related fields.

[0010] In one implementation of the present application, if it does not exist, the document type and document related fields are saved, specifically including: establishing a data integration middle layer to store multi-source document data; mapping and integrating the multi-source document data, and performing consistency verification on the integrated multi-source document data.

[0011] In one implementation of the present application, after generating a graph based on the nodes, the method further includes: sending the graph, document type and document-related fields to the front end; updating and saving according to the modifications of the front end, and checking the modification path according to the data source.

[0012] In one implementation of the present application, a graph is generated based on nodes, specifically including: obtaining the document data of the next-level nodes one by one according to the document association fields and the ID of the first-level node, and displaying them in the graph; if the query shows that the current node has no document data, it is marked with field 0.

[0013] In one implementation of the present application, the method also includes: when any document data changes, updating the document data of the corresponding node in real time or according to a preset time period, and displaying it in a graph; based on the document data including urgent documents and non-urgent documents, asynchronous calculation is used for non-urgent documents.

[0014] In one implementation of the present application, the method also includes: setting data access permissions, providing a user interaction interface for those with access permissions, allowing users to drag, scale and zoom in on the graph; encrypting document data transmission and storage, and recording operations on the graph.

[0015] In a second aspect, an embodiment of the present application further provides a device for dynamically capturing and displaying multi-source data, the device comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: connect to multiple systems and capture multi-source document data in real time, the document data comprising tables and images; perform text recognition on the document data through natural language processing to obtain the document type and document-related fields; obtain the upstream field and downstream field of the document name through regular expressions based on the document-related fields; obtain a first-level node based on the document name, and configure an ID for the first-level node; search for document data whose upstream field is a first-level node to obtain a second-level node; generate a graph based on the nodes, with the first-level node taking the center position of the canvas as the node position, and the second-level nodes being evenly distributed 360° around the first-level node.

[0016] On the third aspect, the embodiments of the present application also provide a non-volatile computer storage medium for dynamically capturing and displaying multi-source data, storing computer executable instructions, and the computer executable instructions are configured to: connect to multiple systems and capture multi-source document data in real time, the document data including tables and images; perform text recognition on the document data through natural language processing to obtain the document type and document-related fields; obtain the upstream field and downstream field of the document name through regular expressions based on the document-related fields; obtain the first-level node based on the document name, and configure an ID for the first-level node; search for document data whose upstream field is the first-level node to obtain the second-level node; generate a graph based on the nodes, with the first-level node taking the center position of the canvas as the node position, and the second-level nodes being evenly distributed 360° around the first-level node.

[0017] The embodiments of the present application provide a method, device and medium for dynamically capturing and displaying multi-source data, which connects multiple systems to achieve data sharing and reduce errors and redundancy caused by data islands; the centralized display and visualization of data can intuitively show the relationship and interaction between data, and the generated graph can help users better understand business conditions and trends; data sharing and automated tracking can reduce manual intervention and duplication of work, improve the efficiency of data processing and analysis, and thus speed up business decision-making; data sharing and tracking help enterprises to promptly identify potential risks and problems. In summary, solving the problem of data being unable to be shared, displayed and tracked between multiple systems has far-reaching significance and important value for the long-term development of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A flowchart of a method for dynamically capturing and displaying multi-source data provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of document types for a method for dynamically capturing and displaying multi-source data provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of document-related fields of a method for dynamically capturing and displaying multi-source data provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a method for dynamically capturing and displaying multi-source data provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the internal structure of a device for dynamically capturing and displaying multi-source data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0025] The embodiments of the present application provide a method, device and medium for dynamically capturing and displaying multi-source data, which solves the problem in the prior art that data between multiple systems are interrelated but cannot be shared, displayed and tracked, which limits the association between data and makes it impossible to intuitively see the interaction between data.

[0026] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0027] Figure 1 A flowchart of a method for dynamically capturing and displaying multi-source data provided in an embodiment of the present application. Figure 1 As shown, a method for dynamically capturing and displaying multi-source data provided by an embodiment of the present application specifically includes the following steps:

[0028] Step 10: Connect multiple systems to capture multi-source document data in real time, including forms and images.

[0029] As an optional embodiment, multiple systems are connected to capture document data from multiple sources in real time, which may specifically include: Step 101: polling and monitoring multiple systems, the systems include internal enterprise systems and external enterprise systems; Step 102: when new document data is captured, extract the new document data.

[0030] In this step, the internal system of the enterprise may include an ERP (Enterprise Resource Planning) system, which integrates procurement, sales, inventory, finance, etc. The external system may include an e-commerce platform, which connects the internal system of the enterprise with the external system of the enterprise. Polling and monitoring different business systems will generate various types of document data.

[0031] Step 20: Perform text recognition on the document data through natural language processing to obtain the document type and document-related fields;

[0032] In this step, if Figure 2 and Figure 3 As shown, by analyzing the keywords and context information in the text, the type of document and the document-related fields of the document can be identified.

[0033] As an optional embodiment, text recognition is performed on document data through natural language processing to obtain document type and document-related fields, which may specifically include: Step 201: pre-process new document data and extract text from tables and images; Step 202: identify and obtain document type and document-related fields through natural language processing, where the document type includes the document name and data source; Step 203: determine whether the document type and document-related fields exist in the database; Step 204: if not, save the document type and document-related fields.

[0034] In this step, if the document type and document related fields exist in the database, they can be matched directly, for example Figure 2 As shown, the document name is Project Approval and the data source is Project Approval Form.

[0035] As an optional embodiment, if it does not exist, saving the document type and document related fields may specifically include: Step 2041: establishing a data integration middle layer to store multi-source document data; Step 2042: mapping and integrating the multi-source document data, and performing consistency check on the integrated multi-source document data.

[0036] In this step, data mapping is performed on the document data in each source system, the corresponding relationship between data fields is clarified, and the source data is converted, including renaming of data fields, conversion of data types, unification of data units, etc.

[0037] Step 30: According to the document related fields, obtain the upstream field and downstream field of the document name through regular expressions;

[0038] In this step, if Figure 3 As shown in the figure, the upstream document related field is project approval, and the downstream document related field is engineering service bidding form.

[0039] Step 40: Get the first-level node according to the document name and configure the ID for the first-level node;

[0040] In this step, if Figure 2 As shown in the figure, project establishment is determined as a first-level node. In actual operation, multiple different upstream document types can correspond to the same business downstream document type, or multiple different downstream document types can correspond to the same business upstream document type.

[0041] Step 50: Search for the document data whose upstream field is a first-level node and obtain a second-level node;

[0042] Step 60: Generate a graph based on the nodes. The first-level nodes are located at the center of the canvas, and the second-level nodes are evenly distributed around the first-level nodes at 360 degrees. Figure 3 shown.

[0043] As an optional embodiment, generating a graph based on nodes may specifically include: Step 601: According to the document association field and the ID of the first-level node, obtain the document data of the next-level node one by one, and display it in the graph; Step 602: If it is queried that the current node has no document data, mark it with field 0.

[0044] As an optional embodiment, after generating a graph based on the nodes, the method may further include: Step 70: sending the graph, document type and document-related fields to the front end; Step 80: updating and saving according to the modifications made by the front end, and checking the modification path according to the data source.

[0045] For example, Figure 4 As shown, the uploader will carefully review the structure of the graph, node distribution, classification accuracy of document types, and recognition accuracy of document-related fields. If the recognition accuracy of document types is not high, it may be necessary to optimize the natural language processing algorithm to improve the accuracy of text recognition; for the extraction of related fields, if the regular expression cannot accurately match all situations, it is necessary to adjust the regular expression according to the actual data to ensure that all possible field formats can be covered; for example, use a large amount of classified and labeled document data to train the NLP model; after adjustment, it is necessary to continuously monitor system performance and data processing accuracy to detect and solve problems in a timely manner.

[0046] As an optional embodiment, the method may also include: when any document data changes, updating the document data of the corresponding node in real time or according to a preset time period, and displaying it in a graph; based on the document data including urgent documents and non-urgent documents, asynchronous calculation is used for non-urgent documents.

[0047] In this step, the graph is updated asynchronously to reduce the synchronization burden of the system and improve the response speed. When the data changes, the corresponding events or messages are triggered, and the background service asynchronously processes these events and updates the graph.

[0048] As an optional embodiment, the method may also include: setting data access rights, providing a user interaction interface for those with access rights, allowing users to drag, scale and zoom in on the graph; encrypting document data transmission and storage, and recording operations on the graph.

[0049] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides a device for dynamically capturing and displaying multi-source data, and its structure is as follows: Figure 5 shown.

[0050] Figure 5 A schematic diagram of the internal structure of a device for dynamically capturing and displaying multi-source data provided in an embodiment of the present application. Figure 5 As shown, the device includes:

[0051] at least one processor 501;

[0052] and, a memory 502 communicatively connected to the at least one processor;

[0053] Among them, the memory 502 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 501 so that at least one processor 501 can: connect to multiple systems and capture document data from multiple sources in real time, and the document data includes tables and images; perform text recognition on the document data through natural language processing to obtain the document type and document-related fields; obtain the upstream field and downstream field of the document name through regular expressions based on the document-related fields; obtain the first-level node based on the document name, and configure an ID for the first-level node; search for document data whose upstream field is the first-level node to obtain the second-level node; generate a graph based on the nodes, with the first-level node taking the center position of the canvas as the node position, and the second-level nodes being evenly distributed 360° around the first-level node.

[0054] Some embodiments of the present application provide corresponding Figure 1A non-volatile computer storage medium for dynamically capturing and displaying multi-source data, storing computer executable instructions, wherein the computer executable instructions are configured to: connect to multiple systems, and capture multi-source document data in real time, wherein the document data includes tables and images; perform text recognition on the document data through natural language processing to obtain the document type and document-related fields; obtain the upstream field and downstream field of the document name through regular expressions based on the document-related fields; obtain the first-level node based on the document name, and configure an ID for the first-level node; search for document data whose upstream field is the first-level node to obtain the second-level node; generate a graph based on the nodes, wherein the first-level node is located at the center of the canvas, and the second-level nodes are evenly distributed 360° around the first-level node.

[0055] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the IoT device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0056] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0057] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0058] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0061] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0062] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0063] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0064] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0065] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for dynamically capturing and displaying multi-source data, characterized in that: The method comprises: Connect multiple systems, capture document data from multiple systems in real time, and obtain multi-source document data, including tables and images; Performing text recognition on the document data through natural language processing to obtain the document type and document-related fields; According to the document associated fields, the upstream field and the downstream field of the document name are obtained by using a regular expression; Obtain a first-level node according to the document name, and configure an ID for the first-level node; Searching for the document data whose upstream field is the first-level node to obtain the second-level node; A graph is generated according to the nodes, wherein the first-level node is located at the center of the canvas, and the second-level nodes are evenly distributed around the first-level node at 360°.

2. The method for dynamically capturing and displaying multi-source data according to claim 1, characterized in that: The said connecting multiple systems and capturing the document data of multiple systems in real time specifically includes: Conducting polling and monitoring of multiple systems, including internal and external enterprise systems; When it is detected that new document data is generated, the new document data is extracted.

3. The method for dynamically capturing and displaying multi-source data according to claim 2, characterized in that: The text recognition of the document data is performed by natural language processing to obtain the document type and document-related fields, specifically including: Preprocessing the new document data to extract text from the table and image; By natural language processing, the document type and document-related fields are identified, wherein the document type includes the document name and data source; Determine whether the document type and document-related fields exist in the database; If it does not exist, save the document type and document related fields.

4. The method for dynamically capturing and displaying multi-source data according to claim 3, characterized in that: If it does not exist, save the document type and document related fields, including: Establishing a data integration middle layer to store the multi-source document data; The multi-source document data are mapped and integrated, and consistency check is performed on the integrated multi-source document data.

5. The method for dynamically capturing and displaying multi-source data according to claim 1, characterized in that: After generating a graph according to the nodes, the method further includes: Sending the graph, document type and document related fields to the front end; Update and save according to the changes made on the front end, and check the modification path according to the data source mentioned.

6. The method for dynamically capturing and displaying multi-source data according to claim 1, characterized in that: The generating of the graph according to the node specifically includes: According to the document association field and the ID of the first-level node, the document data of the next-level nodes are obtained one by one, and displayed in the graph; If the query shows that there is no document data in the current node, it will be marked with field 0.

7. The method for dynamically capturing and displaying multi-source data according to claim 6, characterized in that: The method further comprises: When any of the document data changes, the document data of the corresponding node is updated in real time or according to a preset time period, and displayed in the graph; Based on the fact that the document data includes urgent documents and non-urgent documents, asynchronous calculation is adopted for the non-urgent documents.

8. The method for dynamically capturing and displaying multi-source data according to claim 1, characterized in that: The method further comprises: Set data access permissions and provide a user interface for those with access permissions, allowing users to drag, zoom and enlarge the graph; The document data transmission and storage are encrypted, and the operation records of the map are recorded.

9. A device for dynamically capturing and displaying multi-source data, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Connect multiple systems to capture multi-source document data in real time, including forms and images; Performing text recognition on the document data through natural language processing to obtain the document type and document-related fields; According to the document associated fields, the upstream field and the downstream field of the document name are obtained by using a regular expression; Obtain a first-level node according to the document name, and configure an ID for the first-level node; Searching for the document data whose upstream field is the first-level node to obtain the second-level node; A graph is generated according to the nodes, wherein the first-level node takes the center of the canvas as the node position, and the second-level nodes are evenly distributed around the first-level node at 360°.

10. A non-volatile computer storage medium for dynamically capturing and displaying multi-source data, storing computer executable instructions, characterized in that: The computer executable instructions are configured to: Connect multiple systems to capture multi-source document data in real time, including forms and images; Performing text recognition on the document data through natural language processing to obtain the document type and document-related fields; According to the document associated fields, the upstream field and the downstream field of the document name are obtained by using a regular expression; Obtain a first-level node according to the document name, and configure an ID for the first-level node; Searching for the document data whose upstream field is the first-level node to obtain the second-level node; A graph is generated according to the nodes, wherein the first-level node takes the center of the canvas as the node position, and the second-level nodes are evenly distributed around the first-level node at 360°.

Images