A question and answer method, apparatus, computer device and program product

Abstract

The present disclosure provides a question and answer method, device, computer equipment and program product, wherein the method comprises: constructing a database matched with a data type of batch data to be inputted and a data acquisition mode matched with the data type, and constructing a data association relationship between the batch data; the data type comprises a structured data type and an unstructured data type; performing intent classification and entity extraction on a target question inputted by a user to determine a query intent and a query entity of the target question; determining a target data type matched with the target question according to the query intent and the query entity; acquiring a plurality of target data matched with the target question from the database according to the target data type and the data association relationship; and determining an answer result matched with the target question according to the plurality of target data.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a question-answering method, apparatus, computer device, and program product. Background Art

[0002] In the context of the convergence of big data and artificial intelligence, users are increasingly demanding and complex ways to use AI models in conjunction with a company's private big data for question-and-answer interactions. For example, professionals in the financial industry need to use AI models to conduct compliance reviews of financial operations, combining them with a company's financial transaction data and risk assessment documents. Similarly, professionals in the healthcare industry need to integrate medical records with diagnostic and treatment guidelines to aid clinical decision-making.

[0003] However, the current question-and-answer interaction method requires users to actively upload the data they need to the artificial intelligence model before conducting question-and-answer interaction, and there are strict restrictions on the amount of data that users can upload. This not only increases the complexity of the question-and-answer process and affects the user experience, but also there is the problem that the question-and-answer results fed back to users do not meet their expectations due to the limited data that can be used. The disadvantages are very obvious. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide a question-answering method, apparatus, computer device, and program product to reduce the complexity of the question-answering process and improve the accuracy of the answer results.

[0005] In a first aspect, an embodiment of the present disclosure provides a question-answering method, comprising:

[0006] According to the data type of the batch data to be entered and the data acquisition method matching the data type, a database matching the data type is constructed, and a data association relationship between the batch data is constructed; the data type includes structured data type and unstructured data type;

[0007] Performing intent classification and entity extraction on the target question input by the user to determine the query intent and query entity of the target question;

[0008] Determining a target data type that matches the target question based on the query intent and the query entity;

[0009] Acquire multiple pieces of target data matching the target question from the database according to the target data type and the data association relationship;

[0010] An answer result matching the target question is determined based on the multiple pieces of target data.

[0011] In a possible implementation, the step of constructing a database matching the data type according to the data type of the batch data to be entered and a data acquisition method matching the data type, and constructing a data association relationship between the batch data, includes:

[0012] In a case where the data type includes a structured data type, determining that the data acquisition method includes a data source connection method;

[0013] Connecting the data source of the structured batch data according to the data source connection mode, obtaining the batch data from the connected data source, and extracting each data field of the batch data;

[0014] Based on the acquired batch data and the extracted data fields, a structured database corresponding to the structured data type is constructed, and according to the correlation between the data fields of the batch data, a data association relationship between the batch data is constructed.

[0015] In a possible implementation, the step of constructing a database matching the data type according to the data type of the batch data to be entered and the data acquisition method matching the data type, and establishing data association relationships between the batch data, further includes:

[0016] In a case where the data type includes an unstructured data type, determining that the data acquisition method includes an interface upload method;

[0017] According to the interface upload method, each data folder uploaded by the user and the application scenario of each data folder are obtained through the file upload interface; the data folder includes batch files;

[0018] Determining a file vector for each unstructured file in the batch file based on parallel parsing of the batch files in each of the data folders, and constructing a file index corresponding to the batch file according to the file vector and the application scenario;

[0019] Constructing an unstructured database corresponding to the unstructured data type according to the file index and the file vector;

[0020] A data association relationship between batch files is constructed based on the file vectors, and a data association relationship between the batch files and the batch data is constructed based on the file vectors and data fields.

[0021] In a possible implementation, obtaining a plurality of target data matching the target question from the stored batch data according to the target data type and the data association relationship includes:

[0022] screening, from a sub-database in the database that matches the target data type, a plurality of initial data matching the target question based on a data update time of each piece of database data in the sub-database, a first similarity between each piece of database data, and a second similarity between each piece of database data and the target question;

[0023] Based on the data association relationship of each piece of the initial data, acquiring associated data related to the initial data;

[0024] A third similarity between the associated data and the target question is determined, and based on the third similarity and the second similarity, a plurality of target data matching the target question is determined from the initial data and the associated data.

[0025] In a possible implementation, determining an answer result matching the target question based on the multiple pieces of target data includes:

[0026] generating at least one entity classification under a target query entity based on the plurality of target data and the query entity; the entity classification is used to be displayed to a user; the target query entity includes at least part of the query entity;

[0027] Based on the target entity classification selected by the user under various target query entities, determining matching data matching the target entity classification from the target data;

[0028] Based on the matching data, an answer result matching the target question is generated.

[0029] In a possible implementation, determining an answer result matching the target question based on the multiple pieces of target data includes:

[0030] If no answer can be determined based on the plurality of target data, determining whether the user authorizes the online search;

[0031] If yes, obtaining network data matching the target question based on the multiple target data and the target question;

[0032] An answer result matching the target question is generated based on the network data and the target data.

[0033] In a possible implementation, generating an answer result matching the target question based on the network data and the target data includes:

[0034] Filtering the network data according to the data update time of the target data and the data field or file index corresponding to the target data to obtain filtered network data;

[0035] Storing the filtered network data in a database that matches the unstructured data type, and determining whether to use the unstructured data type as a new target data type based on the target data type;

[0036] If so, the unstructured data type is used as a new target data type, and the step of obtaining multiple target data matching the target question from the database based on the target data type and the data association relationship is returned until an answer result matching the target question is obtained.

[0037] In a possible implementation, the structured batch data includes a structured data table, a streaming data table, and a file table;

[0038] The data source includes a structured database corresponding to the structured data table, a streaming data source corresponding to the streaming data table, and a file system data source corresponding to the file table.

[0039] In a second aspect, an embodiment of the present disclosure further provides a question-answering device, comprising:

[0040] A construction module is used to construct a database matching the data type according to the data type of the batch data to be entered and the data acquisition method matching the data type, and to establish a data association relationship between the batch data; the data type includes a structured data type and an unstructured data type;

[0041] The first determination module is used to perform intent classification and entity extraction on the target question input by the user, and determine the query intent and query entity of the target question;

[0042] A second determination module is used to determine a target data type that matches the target question according to the query intent and the query entity;

[0043] An acquisition module, configured to acquire, from the database, a plurality of target data matching the target question according to the target data type and the data association relationship;

[0044] The third determination module is used to determine an answer result that matches the target question based on the multiple target data.

[0045] In a third aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0046] In a fourth aspect, an optional implementation of the present disclosure further provides a computer program product, including a computer program, which, when executed, implements the above-mentioned first aspect, or the steps in any possible implementation of the first aspect.

[0047] The question-and-answer method, apparatus, computer equipment, and program product provided by the embodiments of the present disclosure can achieve batch access to data by constructing a database that matches batch data of different data types, and can improve the utilization rate of data in the database by constructing data association relationships between batch data. Based on the batch-accessed data, the user does not need to manually upload data, but can achieve question-and-answer interaction by simply inputting the target question, thereby reducing the complexity of the question-and-answer interaction and improving the user's question-and-answer experience. In addition, by automatically analyzing the target question, the query intent and query entity are determined, and then the target data type is determined based on the query intent and query entity, which can narrow the scope of the database required to query the target question, and then query in combination with the data association relationship, which can improve the rationality and accuracy of the queried target data, thereby generating answers that are more in line with user expectations and improving the accuracy of the question-and-answer interaction.

[0048] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0050] Figure 1 A flowchart of a question-answering method provided by an embodiment of the present disclosure is shown;

[0051] Figure 2 It shows a data entry page corresponding to a structured database provided by an embodiment of the present disclosure;

[0052] Figure 3 It shows a data entry page corresponding to another structured database provided by an embodiment of the present disclosure;

[0053] Figure 4 It shows a data entry page corresponding to another structured database provided by an embodiment of the present disclosure;

[0054] Figure 5 A schematic diagram of a display page for obtaining batch folders provided by an embodiment of the present disclosure is shown;

[0055] Figure 6 A schematic diagram of a question-and-answer interaction page provided by an embodiment of the present disclosure is shown;

[0056] Figure 7 A schematic diagram of a question-answering device provided by an embodiment of the present disclosure is shown;

[0057] Figure 8 A schematic structural diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0059] In addition, the terms "first," "second," and the like in the description and claims of the embodiments of the present disclosure and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments described herein can be practiced in an order other than that shown or described herein.

[0060] In this document, "multiple or several" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0061] Research has found that when current AI big models engage in question-and-answer interactions with users, users must actively upload their private data and submit questions through the model's application programming interface (API). The big model then generates answers based on the uploaded data. However, due to the limited amount of data that can be uploaded and the need for users to actively specify the data, current big models are unable to address the following issues: 1. Data silos: Structured and unstructured data cannot be integrated in bulk, resulting in limited data availability for big models to answer user questions and inability to fully utilize the full data set; 2. Inefficient interactions: Users must manually specify data sources (e.g., "Please query table XX" or "Please refer to document XX"), leaving the big model with limited independent judgment. These issues ultimately prevent the big model from generating answers that meet user expectations, hindering effective interactions.

[0062] Based on the above research, the present disclosure provides a question-answering method, apparatus, computer equipment and program product. By constructing a database that matches batch data of different data types, batch access to data can be achieved, and by constructing data association relationships between batch data, the utilization rate of data in the database can be improved. Based on batch-accessed data, question-answering interaction can be achieved by simply inputting target questions without the user having to manually upload data, thereby reducing the complexity of question-answering interaction and improving the user's question-answering experience. In addition, by automatically analyzing the target question, the query intent and query entity are determined, and then the target data type is determined based on the query intent and query entity. This can narrow the scope of the database required to query the target question, and then query in combination with the data association relationship to improve the rationality and accuracy of the queried target data, thereby generating answers that are more in line with user expectations and improving the accuracy of question-answering interaction.

[0063] The defects in the above solutions are the results obtained by the inventors after practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by this disclosure for the above problems below should be the contributions made by the inventors to this disclosure during the disclosure process.

[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0065] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0066] It should be noted that the specific terms mentioned in the embodiments of the present disclosure include:

[0067] MySQL: is a relational database management system that can store data in different database tables, thereby increasing data access speed and flexibility;

[0068] Hive: A data warehouse infrastructure based on Hadoop that maps structured data files into a database table and uses simple Structured Query Language (SQL) statements to query and analyze the stored data.

[0069] ORACLE: A widely used database management system. As a general-purpose database system, it has complete data management capabilities. As a relational database, it is a complete relational product. As a distributed database, it implements distributed processing capabilities.

[0070] Kafka: is a high-throughput distributed publish-subscribe messaging system that can handle all the action stream data of consumers in the website;

[0071] Structured data: data stored in a fixed format, such as data tables in relational databases (including table structure, field definitions, and data types), Excel spreadsheets, CSV files, etc. The characteristic is that the data can be logically expressed in a two-dimensional table format.

[0072] Unstructured data: Data that does not follow a fixed format, such as Portable Document Format (PDF) documents, Word documents, plain text files, images, audio, and video. Key information must be extracted through technologies such as natural language processing and image recognition.

[0073] DB2: A relational database management system that primarily runs on UNIX. It's primarily used in large-scale applications and offers good scalability, supporting environments from mainframes to single-user environments. It's used on all common server operating system platforms.

[0074] OceanBase: A distributed database with a proprietary integrated architecture that combines the scalability of a distributed architecture with the performance advantages of a centralized architecture. It features strong data consistency, high availability, high performance, online scalability, high compatibility with SQL standards and mainstream relational databases, and low cost.

[0075] Postgres: PostgreSQL is a full-featured free software object-relational database management system.

[0076] GoldenDB: is a financial-grade transactional distributed database.

[0077] To facilitate understanding of this embodiment, a question-and-answer method disclosed in an embodiment of the present disclosure is first introduced in detail. The executor of the question-and-answer method provided in the embodiment of the present disclosure is generally a terminal device or other processing device with certain computing capabilities, where the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant (PDA), a handheld device, a computer device, etc.; in some possible implementation methods, the question-and-answer method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0078] The question-and-answer method provided in the embodiment of the present disclosure is described below by taking a computer device as an example.

[0079] like Figure 1 FIG. 1 is a flowchart of a question-answering method provided by an embodiment of the present disclosure, which may include the following steps:

[0080] S101: According to the data type of the batch data to be entered and the data acquisition method that matches the data type, a database that matches the data type is constructed, and a data association relationship between the batch data is constructed; the data type includes a structured data type and an unstructured data type.

[0081] Here, the batch data to be entered is large-scale data to be entered into the data management system. The batch data may include structured batch data and unstructured batch unstructured data. The structured data may include, for example, structured data tables, streaming data tables, and file tables.

[0082] A batch structured data table can be a table stored in a structured database, such as a database table in MySQL, Hive, Oracle, or other structured databases. A streaming data table can be a data table corresponding to batch real-time data provided by a streaming data source, such as a Kafka streaming data table. A file table can be a data table stored in a file system, such as an Excel file or a comma-separated values ​​(CSV) file.

[0083] Databases that match data types may include structured databases that match structured data types and unstructured databases that match unstructured data types. In the embodiments of the present application, data of different data types correspond to different data acquisition methods, and the data acquisition methods are used to at least access and acquire batch data. By providing data acquisition methods that match different data types, the present application can integrate and access batches of structured data (such as millions of data tables in a MySQL database) and batches of unstructured data (such as tens of thousands of business documents within an enterprise), thereby providing matching answer data for the target questions input by the user.

[0084] Data association relationships are used to indicate the association between data. In this application, data association relationships can include association relationships between batches of structured data, association relationships between batches of unstructured data, and association relationships between batches of structured data and batches of unstructured data. For example, if objects A to C are mentioned in the unstructured data, the batches of unstructured data corresponding to objects A to C can be extracted from the unstructured data, and then the batches of structured data corresponding to objects A to C can be extracted from the structured data to establish a data association relationship between the batches of unstructured data corresponding to objects A to C and the batches of structured data corresponding to A to C. In this way, after querying the structured data (or unstructured data) of objects A to C, the unstructured data (or structured data) can be found based on the data association relationship, and the data can be used to generate answers for the user.

[0085] This application addresses the problems of insufficient data access capabilities in the existing technology (such as the inability to access structured databases in batches (such as directly connecting to MySQL and Oracle databases to obtain full table data), the limitation of unstructured data processing to single documents, and the inability to establish cross-document knowledge associations (such as integrating 1,000 product manuals to analyze business problems)). It provides a method for pre-accessing batch data, supports the batch access and dynamic management of massive structured data (such as massive MySQL tables) and massive unstructured documents in private databases, and thus provides users with more expected answer results based on the accessed batch data when interacting with users in question and answer.

[0086] In specific implementations, batch data of various data types that need to be entered can be submitted in advance to the data management system of this application. For each data type of batch data, the batch data can be acquired using a data acquisition method that matches the data type. Based on data analysis of the batch data, data associations between different data are determined, and then the batch data and the determined data associations are stored in a database that matches the data type.

[0087] In one embodiment, since data types may include structured data types and unstructured data types, the access process of batch data of different data types will be specifically introduced below for illustration.

[0088] In the case where the data type includes a structured data type, the above S101 may be implemented according to the following steps A1 to A3:

[0089] A1: When the data type includes a structured data type, determining the data acquisition method includes a data source connection method.

[0090] For example, for bulk data of a structured data type, the bulk data of the structured type can be obtained by connecting to a data source corresponding to the data.

[0091] When the structured bulk data includes a structured data table, the data source may include a structured database corresponding to the structured data table; when the structured bulk data includes a streaming data table, the data source may include a streaming data source corresponding to the streaming data table; when the structured bulk data includes a file table, the data source may include a file system data source corresponding to the file table.

[0092] A2: Connect to the data source of structured batch data according to the data source connection method, obtain the batch data from the connected data source, and extract each data field of the batch data.

[0093] Here, the data field is an attribute field of structured batch data, which can indicate information such as data attributes, data description, field type, business meaning, and data examples of the batch data.

[0094] Because different structured data has different corresponding storage locations, docking methods, and storage methods, different structured data also requires different corresponding data source connection methods. Specifically, for batch data submitted by users for entry, the corresponding data source connection method can be determined based on information such as the storage location, docking method, and storage method of the batch data. Then, according to the data source connection method, the corresponding data source of the batch data is connected. The batch data stored in the connected data source is automatically pulled from the connected data source, and data fields are extracted from the pulled batch data to obtain the individual data fields of the batch data.

[0095] For example, when the data to be entered includes structured database tables in databases such as MySQL, Hive, and Oracle, the user can submit the private database to be connected through the data entry page provided by the data management system provided by this application. Then, the backend of the data management system can connect to these private databases through a matching data acquisition method. After the connection is successful, all data tables and field descriptions stored therein can be automatically pulled in batches and entered into the data management system. Figure 2 As shown, this is a data entry page corresponding to a structured database provided in an embodiment of the present application. This page can be used to enter batch data in data tables of various databases stored offline. The data source type selection box is used to select the database type corresponding to the data to be entered in batches, such as Hive, Oracle, Mysql, kafka, DB2, OceanBase, Postgres, GoldenDB, etc. The data source name box is used to enter the name of the data source; the Hvie metastore box is used to enter the metastore information of the Hive database; the Java Database Connectivity (JDBC) uniform resource locator (URL) box is used to enter the address of the database; the authentication method is used to enter the authentication method of the database; the username and password box is used to enter the database login account and password; the data source remark box is used to enter the remark information of the data source; the test data source button is used to test the data source to be entered to prevent data source anomalies; the connect data source button is used to confirm the entered data source. Users can Figure 2The page contains information related to the structured data tables that require batch entry. Upon successful submission, the backend of the data management system can connect to these private databases using a matching data acquisition method and retrieve the structured data tables and the data fields stored therein, including field descriptions. By entering the MySQL address and account password, the backend can automatically connect to the private database and batch-retrieve all data tables and field descriptions, entering them into the structured database within the data management system.

[0096] For example, when the data to be entered includes a streaming data table in a Kafka streaming data source, the user can also submit the Kafka streaming data source to be connected through the data entry page provided by the data management system provided by this application. Then, the background of the data management system can connect to the Kafka streaming data source. After the connection is successful, all the streaming data and data fields (such as subject, type, attribute, etc.) stored therein can be automatically pulled in batches and entered into the data management system. Figure 3 As shown, this is a data entry page corresponding to another structured database provided in an embodiment of the present application. This page can be used to enter Kafka streaming data tables. The data source type selection box is used to select the database type corresponding to the data to be entered in batches, such as Hive, Oracle, MySQL, Kafka, DB2, OceanBase, Postgres, GoldenDB, etc. The data source name box is used to enter the name of the data source; the table name box and the table description box are used to enter the table name and description information after entering the Kafka streaming data table respectively; the topic box is used to enter the data topic in the streaming data table. There can be multiple topics in the streaming data table, and different topics correspond to different streaming data; the consumer format box is used to enter the font format corresponding to the consumption of the Kafka streaming data table; the character encoding box is used to enter the encoding method for the streaming data; the field box can be used to enter new fields corresponding to the Kafka streaming data table; the field type box is used to enter the field data type; the comment box is used to enter comment information for the new field; the add field button is used to add a new field; the processing time and field value time are used to indicate the time column information of the streaming data; and the create button is used to confirm the entry of the streaming data. Users can Figure 3 Submit various information related to the streaming data tables that need to be batch-entered on this page. After successful submission, the data management system's backend can connect to these private databases using the corresponding data acquisition method and pull the streaming data tables and the various data fields stored in them. This way, for Kafka streaming data sources, by connecting to the streaming data source, batch-enter streaming data tables into the structured database in the data management system.

[0097] For example, when the data to be entered includes Excel / CSV files, the user can also submit the file system data source to be connected through the data entry page provided by the data management system provided by this application, thereby realizing the batch import of file data sets, and automatically identifying the data fields such as numeric, date, and text in the file, generating a data dictionary (such as the Chinese name of the field, business meaning, data examples, etc.) and entering it into the data management system. Figure 4 As shown, this is a data entry page corresponding to another structured database provided in an embodiment of the present application. This page can be used to enter Excel / CSV and other file tables. The data source selection box is used to select the file system data source corresponding to the file table that needs to be entered in batches; the table name box and the table description box are used to enter the table name and description information after the entered file table, respectively; the table name box is used to enter the type of the entered file table; the file format box is used to enter the file format of the entered file table; the CSV column delimiter is used to enter the delimiter column for pulling data from the CSV file; the CSV disable quotes button is used to indicate the input of quoted data; the CSV ignore parsing errors button is used to indicate whether errors in the parsing process are ignored; the CSV allow comments button is used to indicate whether comments are allowed on the data in the CSV file; the CSV array element delimiter box is used to enter the delimiter of the array in the CSV file; the file path box is used to enter the storage path of the entered file table; the character encoding box is used to enter the encoding method for the data in the file table; the field box can be used to enter the new field corresponding to the file table; the field type box is used to enter the field data type of the new field; the comment box is used to enter the comment information of the new field; the create button is used to confirm the entry of the file table. Users can Figure 4 The page contains information related to the file table that needs to be batch-entered. After successful submission, the backend of the data management system can connect to the file system data source and pull the file table and the various data fields stored in it. This allows batch entry of data from Excel / CSV files into the structured database within the data management system through the file system data source.

[0098] It should be noted that in Figures 2-4 In the " are the component boxes that users must enter.

[0099] A3: Based on the acquired batch data and extracted data fields, a structured database corresponding to the structured data type is constructed. Based on the correlation between the data fields in the batch data, data association relationships between the batch data are constructed.

[0100] In a specific implementation, after pulling batch data and various data fields of the batch data from the data source input by the user, if a structured database already exists, the batch data and various data fields of the batch data can be stored in the already constructed structured database. If a structured database does not exist, a structured database needs to be created in the data management system, and then the pulled batch data and various data fields are stored in the structured database. At the same time, based on the correlation between the data fields of each data in the batch data, the associated data can be determined, and the data association relationship between these data can be constructed, and these data association relationships can also be stored in the structured database.

[0101] In another embodiment, when the data type includes an unstructured data type, the above S101 may be implemented according to the following steps B1 to B5:

[0102] B1: When the data type includes a structured data type, determining that the data acquisition method includes an interface upload method.

[0103] This application supports bulk uploading of large amounts of data to this data management system via folders through an interface connection, such as uploading a "Product Documents" folder containing several PDF files. Providing a file upload interface allows users to upload bulk unstructured data to the data management system in advance and store it in an unstructured database. Therefore, if the data type includes structured data types, the data acquisition method will include an interface upload method.

[0104] For example, when it is determined that batches of unstructured data need to be accessed in advance, it can be determined that the data acquisition method includes an interface upload method.

[0105] B2: According to the interface upload method, each data folder uploaded by the user and the application scenario of each data folder are obtained through the file upload interface; the data folder includes batch files.

[0106] Here, a data folder may include batches of files, but this application does not specifically limit the number of batches of files. For example, a data folder may include hundreds or thousands of files of different types. The file types here can be, for example, PDF, Word, image, video, audio, etc. The file upload interface can support the simultaneous upload of one or more data folders.

[0107] Application scenarios are used to indicate the question-and-answer scenarios in which the files in the data folder can be applied. For example, application scenarios may include database indicator scenarios, medical scenarios, financial scenarios, life service scenarios, game scenarios, software development and application scenarios, etc.

[0108] In specific implementation, users can use the unstructured data entry page provided by the data management system of this application to call the file upload interface and submit the various data folders to be uploaded. After the submission is completed, the backend of the data management system can obtain the various data folders uploaded by the user through the file upload interface. In addition, when uploading data folders, users can specify the application scenarios of each data folder.

[0109] like Figure 5 As shown, this is a schematic diagram of a display page for a batch folder obtained provided by the present application, wherein the identity document (ID) is used to indicate the identifier of the data folder; the name is used to indicate the name of the data folder; the document status is used to indicate the current status of the data folder; the type is used to indicate the folder type of the data folder; the source is used to indicate the source of the data folder, Figure 5 Each data folder in the file is uploaded by the user; the associated scenario is the application scenario of the data folder entered by the user; the creation time indicates the upload time of the data folder; the creator indicates the name of the user who uploaded the data file; and the operation button is used to perform various data operations on the data folder. The file name, type, and scenario boxes are used to filter data folders.

[0110] B3: Based on the parallel parsing of the batch files in each data folder, the file vector of each unstructured file in the batch files is determined, and according to the file vector and the application scenario, a file index corresponding to the batch files is constructed.

[0111] Here, the file index is used to index a certain number of unstructured files. This application can perform parallel parsing of batch files in multiple data folders uploaded simultaneously by users to obtain file vectors for the batches of unstructured files. The unstructured batch data can specifically be unstructured batch files, and the unstructured batch files can be batches of unstructured files. The file vector is used to represent the file content of the unstructured files.

[0112] In specific implementations, after obtaining each data folder uploaded by the user, each unstructured file in each data folder can be parsed in parallel to obtain a file vector for each unstructured file. Based on the file vector of each unstructured file, the file index corresponding to the unstructured file and the unstructured files associated with each file index are determined. At the same time, based on the application scenario of each data folder, the scenario index of each data folder is determined, and the scenario index of the data folder is also used as the file index of each unstructured file under the data folder file.

[0113] B4: Based on the file index and file vector, construct an unstructured database corresponding to the unstructured data type.

[0114] In specific implementations, after determining each file index and file vector, if an existing unstructured database exists, these file indexes and file vectors for the batch of files can be stored in the existing unstructured database. If an existing unstructured database does not exist, an unstructured database must be created in the data management system, and then these file indexes and file vectors for the batch of files must be stored in the unstructured database.

[0115] B5: Construct data association relationships between batch files based on file vectors, and construct data association relationships between batch files and batch data based on file vectors and data fields.

[0116] In specific implementations, the correlation between unstructured files can be calculated based on their file vectors. Based on this correlation, highly correlated unstructured files can be identified and corresponding data association relationships can be constructed. Alternatively, the correlation between unstructured files within the same file index can be determined based on their file vectors. Based on this correlation, highly correlated unstructured files can be identified and corresponding data association relationships can be constructed. This approach can establish knowledge associations across documents (e.g., integrating hundreds or thousands of product manuals to analyze enterprise issues) and generate data association relationships between batches of files.

[0117] At the same time, the correlation between unstructured files and structured data is calculated based on the file vectors of each unstructured file and the data fields of each structured data. Based on this correlation, highly correlated unstructured files are identified and corresponding data association relationships are constructed. In this way, knowledge associations can be established across data types, and data association relationships can be obtained between batches of files and batches of data.

[0118] Then, the data association relationship between the batch files and the data association relationship between the batch files and the batch data can be stored in an unstructured database, and the data association relationship between the batch files and the batch data can be stored in a structured database.

[0119] In this way, the batch parsed data folders can be automatically identified and indexed, making it easier to obtain relevant knowledge from the massive document data for subsequent user questions. By building a structured database and an unstructured database, this application can achieve batch access and dynamic association of massive structured data (such as MySQL data) and massive unstructured documents in private databases.

[0120] S102: Perform intent classification and entity extraction on the target question input by the user to determine the query intent and query entity of the target question.

[0121] Here, the present application can also provide a question-answering network model for question-answering interaction, which can be connected to the data management system of the present application to answer user questions based on the data entered in batches in the data management system.

[0122] The target question is the question submitted by the user on the Q&A interactive page corresponding to the Q&A network model. Intent classification is used to determine the query intent of the target question, and entity extraction is used to extract the query entities in the target question.

[0123] The query intent is the question intention corresponding to the target question submitted by the user, and the query entity is the keywords mentioned in the target question.

[0124] In specific implementations, the target question submitted by the user on the Q&A interactive page can be obtained, and then the intent recognition model can be used to classify the target question into its type, thereby determining the query intent corresponding to the target question. For example, question types can include data analysis, data query, and knowledge Q&A. The query intent corresponding to the data analysis type can be data analysis intent, the query intent corresponding to the data query type can be data query intent, and the query intent corresponding to the knowledge question type can be Q&A search intent. For example, if the target question is "Query the order volume in XX region in XX month of XX year," the query intent can be data query.

[0125] At the same time, Named Entity Recognition (NER) technology can be used to segment and identify keywords in the target question, obtaining the keywords in the target question and using these keywords as query entities. For example, if the target question is "Query the order volume in XX region in XX year and XX month," the query entities may include the time entity (XX year and XX month), the region entity (XX region), and the data type entity (order volume, which can correspond to the order number field in a structured database).

[0126] S103: Determine the target data type that matches the target question based on the query intent and query entity.

[0127] Here, the target data type indicates the type of data to be queried to answer the target question; the target data type may include one or more. For example, if the target data type is a structured data type, it may be determined that the data needs to be queried from a structured database; if the target data type is an unstructured data type, it may be determined that the data needs to be queried from an unstructured database.

[0128] For example, a data requirement analysis can be conducted on the target question based on the query intent and query entities to determine the various target data types needed to answer the target question. For example, if the target question contains query entities such as "analysis" and "trend", it can be determined that structured data (for numerical calculations) and unstructured data (for business knowledge interpretation) need to be combined, so the target data types can include structured data types and unstructured data types; that is, it is necessary to query the data for users to answer the target question from both structured and unstructured databases. For another example, if the target question is "Analyze the reasons for the decline in order volume", it can be determined that the order table (data of structured data type) and market research reports (data of unstructured type) are called simultaneously to answer the question, so the target data types can include structured and unstructured data types.

[0129] S104: Acquire multiple target data matching the target question from the database according to the target data type and data association relationship.

[0130] Here, the target data may be a plurality of pieces of data recalled from a database matching the target data type. Since the database constructed in this application includes a large amount of data, in order to avoid recalling too much data, the target data may be a set number of the most relevant data.

[0131] During specific implementation, you can first obtain batch data from a database that matches the target data type, and calculate the first cosine similarity between each data in the batch data and the target question, and use the top n data with the largest similarity as candidate data. The number n can be set based on experience and can be greater than or equal to the set number. If the data is structured data, the cosine similarity between the data field of the structured data and the target question can be used as the first cosine similarity; if the data is unstructured data, the cosine similarity between the file vector of the unstructured data and the target question can be used as the first cosine similarity. Then, for each candidate data, based on the data association relationship of the data, obtain each associated data of the candidate data. Calculate the second cosine similarity between each associated data and the target question. Based on the first cosine similarity corresponding to the candidate data and the second pre-similarity of each associated data, determine the target cosine similarity corresponding to the candidate data.

[0132] For example, the average of the first cosine similarity and each second cosine similarity can be used as the target cosine similarity; alternatively, the target cosine similarity can be determined based on the product of the first cosine similarity, each second cosine similarity, and the similarity between the candidate data and each piece of associated data; alternatively, weights corresponding to the candidate data and the associated data can be determined based on the similarity between the candidate data and the associated data and the data volume of each piece of associated data, and the first cosine similarity and the second cosine similarity can be weighted and summed using the weights to obtain the target cosine similarity. If a candidate data item includes multiple pieces of associated data, weights corresponding to the candidate data and each piece of associated data can be determined. Alternatively, if there are multiple pieces of associated data, a common weight for all associated data items and a weight for the candidate data item can be determined based on the total data volume of all associated data items and the mean / maximum / variance of the second cosine similarities of all associated data items. These weights, along with the first preliminary similarity and the second cosine similarity, can be used to determine the target cosine similarity. Alternatively, if there are multiple pieces of related data, a common weight for all related data and a weight for the candidate data can be determined based on the total amount of all related data and the product of the first m second cosine similarities with the largest second cosine similarities for all related data. These weights, along with the first preliminary similarity and the second cosine similarity, are then used to determine the target cosine similarity. m can be set based on experience and is not specifically limited in this embodiment of the present application.

[0133] If the candidate data has no associated data, the first cosine similarity of the candidate data can be directly used as the target cosine similarity of the candidate data. Then, based on the target cosine similarity of each candidate data piece, a set number of target data pieces can be filtered out from the candidate data and associated data. For example, if the set number is 30, then 30 target data pieces can be filtered out from the candidate data and associated data based on the target cosine similarity.

[0134] Alternatively, the number of associated data items required for the candidate data can be determined based on the product of the second cosine similarities of the associated data items of the candidate data. Then, using the target cosine similarity of the candidate data and the required number of associated data items, a set number of target data items can be filtered out from the candidate data and associated data.

[0135] It is understandable that when using target cosine similarity to filter target data, target candidate data with data similarity greater than a preset similarity can also be determined based on the data similarity between the candidate data. Based on the timestamps of the target candidate data, target candidate data with a time difference from the current time less than the preset time difference is selected as the filtered target data.

[0136] Optionally, if the candidate data includes multiple associated data, the target associated data whose time difference from the current time is less than a set time difference can be filtered out based on the timestamps of the associated data. Then, based on the second cosine similarity of the target associated data and the first cosine similarity of the candidate data, the target cosine similarity of each candidate data can be determined using the target cosine similarity determination method described above. The target cosine similarity can then be used to filter out the target data from the candidate data and the target associated data.

[0137] This allows you to prioritize data within the data type specifically targeted by the question (e.g., "table" for structured data, "document" for unstructured data) by querying the target data from a database containing batches of data matching the target data type. By verifying the relevance and timeliness of the data based on its similarity to the target question and its timestamp, you can improve the accuracy and rationality of the identified target data.

[0138] S105: Determine an answer result that matches the target question based on the multiple target data.

[0139] In practice, a pre-trained question-answering network model can be used to integrate and analyze each target data item, outputting an answer that matches the target question. The answer can then be displayed to the user on the question-answering interactive page.

[0140] In this way, by building a database that matches batch data of different data types, batch access to data can be achieved, and by building data associations between batch data, the utilization rate of the data in the database can be improved. Based on batch access data, question-and-answer interaction can be achieved by simply entering the target question without the user having to manually upload data, reducing the complexity of the question-and-answer interaction and improving the user's question-and-answer experience. In addition, by automatically analyzing the target question, determining the query intent and query entity, and then determining the target data type based on the query intent and query entity, the scope of the database required to query the target question can be narrowed. Querying in combination with data associations can improve the rationality and accuracy of the target data queried, thereby generating answers that better meet user expectations and improving the accuracy of the question-and-answer interaction.

[0141] In one embodiment, the above S104 may be implemented as follows:

[0142] S104-1: Filtering out a plurality of initial data matching the target question from a sub-database in the database that matches the target data type based on a data update time of each database data item in the sub-database, a first similarity between each database data item, and a second similarity between each database data item and the target question.

[0143] Here, the sub-database that matches the target data type is the database used to store data of the target data type. If the target data type is structured, the sub-database can be a structured database; if the target data type is unstructured, the sub-database can be an unstructured database. When storing batch data in a database, the data's timestamp is also stored synchronously. Therefore, the data update time can be the timestamp of the data stored in the database. Database data refers to the individual pieces of data stored in the database, such as structured data stored in a structured database and unstructured data stored in an unstructured database.

[0144] During specific implementation, each piece of database data in the sub-database that matches the target data type can be first obtained, and based on the cosine similarity between the database data, the first similarity between the database data is determined; and based on the cosine similarity between the database data and the target problem, the second similarity between the database data and the target problem is determined. From the database data, similar data with a first similarity greater than a preset similarity are determined, and data with a difference between the data update time of the similar data and the current time greater than the set difference are eliminated to obtain the retained database data. Based on the second similarity of the retained database data and the second similarity of the database data where no similar data exists, these data are sorted in descending order of similarity to obtain a sorting order. The database data with a sorting order less than the target order is used as the initial data. The target order can be set based on experience, and the embodiments of the present application do not specifically limit it. For example, the target order can be related to the set quantity above.

[0145] In this way, by selecting target data based on the data update time, it can be ensured that the latest updated data is used first rather than outdated historical data.

[0146] S104 - 2 : Based on the data association relationship of each piece of initial data, obtain associated data related to the initial data.

[0147] In a specific implementation, for each piece of initial data, various pieces of associated data related to the initial data can be obtained from the structured database and the unstructured database according to the data association relationship of the initial data. If the associated data includes data to be removed, the removed data can be lost.

[0148] S104 - 3 : Determine a third similarity between the associated data and the target question, and determine a plurality of target data matching the target question from the initial data and the associated data based on the third similarity and the second similarity.

[0149] In specific implementations, the cosine similarity between each piece of associated data and the target question can be used as the third similarity. For each piece of initial data, the associated data can be sorted in descending order based on the third similarity, retaining any associated data with a lower sort order than the default. Then, based on the second similarity of the initial data and the third similarity of the retained associated data, the associated data with the initial data that has a predetermined number of similarities before it can be selected as the target data.

[0150] Alternatively, for each initial data item, the number of associated data items to be retained can be determined based on the mean, product, or variance of the third similarities of the associated data items of the initial data item, and target associated data items can be screened from the associated data items based on the retained data items and the third similarities. A set number of target data items can be screened based on the second similarities of the initial data item and the number of target associated data items of the initial data item.

[0151] Alternatively, for each piece of initial data, a target similarity for the initial data can be determined based on the third similarity of each associated data piece of the initial data and the second similarity of the initial data piece. A set number of target data pieces can be selected from the initial data piece based on the target similarity. Furthermore, for each piece of target data piece, all of the associated data pieces of the target data piece can be used as the target data piece, or the associated data pieces of the target data piece whose third similarity is greater than the set similarity piece can be used as the target data piece.

[0152] In one embodiment, the present application can also intervene and interact with the thinking process of the question-answering network model to allow users to obtain answers that are more in line with their expectations. Therefore, the above S105 can also be implemented according to the following steps:

[0153] S105-1: Generate at least one entity classification under the target query entity according to multiple target data and the query entity; the entity classification is used to be displayed to the user; the target query entity includes at least part of the query entity.

[0154] Here, entity classification is used to indicate the different entity types under the query entity. Entity classification can be determined based on the various parts of the target data that match the query entity. For example, if the query entity is "profit" and the target data includes multiple types of profit data, entity classifications may include operating profit, net profit, retained earnings, total revenue, and profit attributable to the parent company. If the query entity is "MN" and the target data includes multiple types of MN-related data, entity classifications may include MN Co., Ltd., MN Services Co., Ltd., and MN Co., Ltd. in Region X. If the query entity is "industry" and the target data includes multiple types of industry data, entity classifications may include sales industry, finance industry, and education industry.

[0155] The target query entity is an entity among the determined query entities for which different types of related entity data exist in the target data. The target query entity may be one or more entities among all query entities.

[0156] In a specific implementation, for each query entity, it is possible to determine whether different types of data related to the query entity exist based on the various pieces of target data related to the query entity. If so, the various existing types can be used as entity categories under the query entity, and the query entity can be used as the target query entity. If not, it can be determined that no entity category exists for the query entity. In this way, each target query entity and entity category that has an entity category can be determined. Each entity category can then be displayed on the question-and-answer interaction page, allowing users to actively select a specific category under each target query entity.

[0157] like Figure 6 As shown, it is a schematic diagram of a question-answering interactive page provided by an embodiment of the present application, wherein the target question input by the user is "How does the average profit of XX Group this year compare with the average profit of the entire industry in Region A and the average profit of each industry in all regions? Help me analyze it and present it with a visual graph"; when the question-answering network model is used to parse the target question and recall a set amount of target data, the target query entity with entity classification can be determined based on the target data, and the entity classification of each target query entity can be displayed to the user. Figure 6 In the example, the target query entities include "Profit," "XX Group," and Industry. Entity categories for Profit include operating profit, net profit, retained earnings, total revenue, and profit attributable to the parent company. Entity categories for XX Group include XX Group Co., Ltd., XX Group Services Co., Ltd., and XX Group Co., Ltd. in Region X. Entity categories for Industry include Sales, Finance, and Education. The user can select or not select the entity categories displayed under each target query entity on the page. When selecting, one or more target entity categories can be selected. Figure 6 The target entity categories selected by the user are "Operating Profit," "XX Group Co., Ltd.", "Sales Industry," and "Financial Industry." Clicking the Confirm button confirms the currently selected category, while the Unconfirm button pauses the analysis process of the question-answering network model.

[0158] S105 - 2 : Based on the target entity classification selected by the user under various target query entities, matching data that matches the target entity classification is determined from the target data.

[0159] Here, the target entity category is the entity category selected by the user among the entity categories displayed under the target query entity; wherein, for any target query entity, the target entity category selected under the target query entity may be one or more, or may be empty.

[0160] In specific implementations, after each entity category is presented to the user, the user can actively select one. Then, based on the target entity category selected by the user under each target query entity, one or more optimal data matching the target entity category can be filtered from the target data and used as the matching data.

[0161] S105-3: Generate an answer result that matches the target question based on the matching data.

[0162] In specific implementation, the question-answering network model can use the matching data under each target query entity, as well as the target data matched by other query entities except the target query entity, to integrate and obtain the answer results that match the target question, and then display the answer results to the user on the question-answering interaction page.

[0163] This allows the optimal data source to be matched based on the user's explicit entity classification. The question-answering network model can then automatically retrieve the optimal data based on the user's selected entity classification, providing answers that better meet the user's expectations. Furthermore, since the answer results are modifiable (i.e., users can choose to intervene in the model screening process to obtain answers that better meet their expectations) and data source selection is automated (i.e., the model automatically selects the database to use without the user having to manually specify the data range), the intelligent interaction experience can be effectively improved.

[0164] In one embodiment, the above S105 may also be implemented according to the following steps T1 to T3:

[0165] T1: If the answer cannot be determined based on multiple target data, determine whether the user authorizes the online search.

[0166] Here, online search is used to authorize searching for data related to the target question from the Internet. Specifically, an online button can be displayed on the question-and-answer interaction page, and the user can authorize or cancel the online search at any time by triggering the online button and the preset trigger mechanism.

[0167] In specific implementation, if an answer cannot be generated based on multiple target data searched from the database, it can be determined whether the user authorizes the online search; if not, a prompt message indicating that the question cannot be answered can be generated and displayed to the user; if so, the following T2 can be executed.

[0168] T2: If yes, then obtain network data that matches the target question based on the multiple target data and the target question.

[0169] Specifically, if the user authorizes a web search, the system can automatically retrieve initial web data matching the multiple target data and target questions from public web data via the network API. The system then parses and cleans the initial web data to obtain web data, which is then integrated into the entire question-and-answer process. Data cleaning can include, for example, removing useless data (such as fake data) and integrating redundant data.

[0170] T3: Generate answer results that match the target question based on network data and target data.

[0171] In practice, the question-answering network model can analyze and integrate the network data and the target data to generate an answer that matches the target question. For example, the answer can be generated by integrating the network data with the target data based on their integration weights.

[0172] In one embodiment, the above step T3 may also be implemented as follows:

[0173] T3-1: Filter the network data according to the data update time of the target data and the data field or file index corresponding to the target data to obtain filtered network data.

[0174] In specific implementations, the authenticity and matching degree of network data can be verified based on the target data's data update time, data fields, or file indexes. The authenticity of the network data and its matching degree with the target data can then be used as filtered network data.

[0175] For example, the network data and target data can be aligned using dimensions such as timestamp and region to determine whether the network data matches the target data. The network data that matches the contract year, region, and data disclosure time of the target data is considered filtered network data.

[0176] T3-2: Store the filtered network data in a database that matches the unstructured data type, and determine whether to add the unstructured data type as a new target data type based on the target data type.

[0177] In specific implementations, sub-data under the data fields or file indexes of the filtered network data can be generated according to the data fields or file indexes, and these sub-data can be temporarily added to the unstructured database that matches the unstructured data type. Then, if the target data type includes the unstructured data type, it is determined that the unstructured data type does not need to be added as a new target data type. If the target data type does not include the unstructured data type, it is determined that the unstructured data type needs to be added as a new target data type.

[0178] By determining whether to use the unstructured data type as a new target data type, it can be ensured that when the target data is reselected later, the filtered network data can be obtained from the unstructured database.

[0179] T3-3: If yes, the unstructured data type is used as a new target data type, and the steps of obtaining multiple target data matching the target question from the database are returned based on the target data type and data association relationship, until an answer matching the target question is obtained.

[0180] In specific implementations, if an unstructured data type is determined as a new target data type, the process can directly return to S104 above to redetermine the target data that matches the target question. Using the question-answer network model, the target data is used to generate a new answer that matches the target question, following the result generation process described above. Here, since the unstructured data type has been added as a new target data type and network data has been supplemented in the unstructured database, after returning to S104 above, the newly queried target data includes at least some of the supplemented filtered network data, thereby ensuring that an answer that matches the target question can be generated.

[0181] In addition, if it is determined that the unstructured data type does not need to be added as a new target data type, it means that the target data type determined at the beginning includes the unstructured data type. Then, after returning to execute the above S104, the re-queried target data includes at least part of the supplemented filtered network data, thereby ensuring that an answer result matching the target question can be generated.

[0182] In addition, when displaying various pages to users, such as question-and-answer interaction pages, data entry pages, and data folder pages, this application can produce pages that match the device type based on the device type used by the user, thereby achieving the purpose of distinguishing device-side characteristics and adapting to various device types.

[0183] In this way, the question-answering network model of this application can proactively interact with the data management system and match the optimal data based on the user's target question, without the user having to actively specify the data they need, thus improving the interactive Q&A experience. Furthermore, users can intervene and interact with the model's thinking process through entity classification, ensuring that they receive answers that better meet their expectations. Furthermore, this application supports batch processing of massive amounts of structured data (e.g., a single table with ≥ 1 million records) and tens of thousands of unstructured documents (e.g., a single import of ≥ 10,000 files), improving data access efficiency. By establishing cross-data type associations (e.g., automatically matching a "product ID" in a table with a "product manual" in a document), data utilization is significantly improved. Enterprise users can use this data management system to quickly build a "private data brain" that accesses all enterprise data and adaptively obtains answers to questions without specifying data scopes. In certain customer service scenarios, the time it takes to answer complex questions is significantly reduced compared to traditional methods of searching for information across different databases, greatly improving the efficiency of data Q&A interactions. It supports offline / online hybrid mode, and gives priority to calling local data in the data management system in a weak network environment to ensure the continuity of interaction.

[0184] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0185] Based on the same inventive concept, a question-and-answer device corresponding to the question-and-answer method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned question-and-answer method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0186] like Figure 7 FIG. 1 is a schematic diagram of a question-answering device provided by an embodiment of the present disclosure, comprising:

[0187] A construction module 701 is configured to construct a database matching the data type and a data acquisition method matching the data type according to the data type of the batch data to be entered, and to establish a data association relationship between the batch data; the data type includes a structured data type and an unstructured data type;

[0188] A first determination module 702 is configured to perform intent classification and entity extraction on a target question input by a user, and determine a query intent and query entity of the target question;

[0189] A second determination module 703 is configured to determine a target data type that matches the target question based on the query intent and the query entity;

[0190] An acquisition module 704 is configured to acquire, from the database, a plurality of target data matching the target question according to the target data type and the data association relationship;

[0191] The third determining module 705 is configured to determine an answer result that matches the target question based on the plurality of target data.

[0192] In a possible implementation, the construction module 701, when constructing a database matching the data type according to the data type of the batch data to be entered and the data acquisition method matching the data type, and establishing a data association relationship between the batch data, is configured to:

[0193] In a case where the data type includes a structured data type, determining that the data acquisition method includes a data source connection method;

[0194] Connecting the data source of the structured batch data according to the data source connection mode, obtaining the batch data from the connected data source, and extracting each data field of the batch data;

[0195] Based on the acquired batch data and the extracted data fields, a structured database corresponding to the structured data type is constructed, and according to the correlation between the data fields of the batch data, a data association relationship between the batch data is constructed.

[0196] In a possible implementation, the construction module 701, when constructing a database matching the data type according to the data type of the batch data to be entered and the data acquisition method matching the data type, and establishing a data association relationship between the batch data, is configured to:

[0197] In a case where the data type includes an unstructured data type, determining that the data acquisition method includes an interface upload method;

[0198] According to the interface upload method, each data folder uploaded by the user and the application scenario of each data folder are obtained through the file upload interface; the data folder includes batch files;

[0199] Determining a file vector for each unstructured file in the batch file based on parallel parsing of the batch files in each of the data folders, and constructing a file index corresponding to the batch file according to the file vector and the application scenario;

[0200] Constructing an unstructured database corresponding to the unstructured data type according to the file index and the file vector;

[0201] A data association relationship between batch files is constructed based on the file vectors, and a data association relationship between the batch files and the batch data is constructed based on the file vectors and data fields.

[0202] In a possible implementation, the acquisition module 704, when acquiring multiple pieces of target data matching the target question from the stored batch data according to the target data type and the data association relationship, is configured to:

[0203] screening, from a sub-database in the database that matches the target data type, a plurality of initial data matching the target question based on a data update time of each piece of database data in the sub-database, a first similarity between each piece of database data, and a second similarity between each piece of database data and the target question;

[0204] Based on the data association relationship of each piece of the initial data, acquiring associated data related to the initial data;

[0205] A third similarity between the associated data and the target question is determined, and based on the third similarity and the second similarity, a plurality of target data matching the target question is determined from the initial data and the associated data.

[0206] In a possible implementation, the third determination module 705, when determining the answer result matching the target question based on the multiple pieces of target data, is configured to:

[0207] generating at least one entity classification under a target query entity based on the plurality of target data and the query entity; the entity classification is used to be displayed to a user; the target query entity includes at least part of the query entity;

[0208] Based on the target entity classification selected by the user under various target query entities, determining matching data matching the target entity classification from the target data;

[0209] Based on the matching data, an answer result matching the target question is generated.

[0210] In a possible implementation, the third determination module 705, when determining the answer result matching the target question based on the multiple pieces of target data, is configured to:

[0211] If no answer can be determined based on the plurality of target data, determining whether the user authorizes the online search;

[0212] If yes, obtaining network data matching the target question based on the multiple target data and the target question;

[0213] An answer result matching the target question is generated based on the network data and the target data.

[0214] In a possible implementation, the third determination module 705, when generating an answer result matching the target question based on the network data and the target data, is configured to:

[0215] Filtering the network data according to the data update time of the target data and the data field or file index corresponding to the target data to obtain filtered network data;

[0216] Storing the filtered network data in a database that matches the unstructured data type, and determining whether to use the unstructured data type as a new target data type based on the target data type;

[0217] If so, the unstructured data type is used as a new target data type, and the step of obtaining multiple target data matching the target question from the database based on the target data type and the data association relationship is returned until an answer result matching the target question is obtained.

[0218] In a possible implementation, the structured batch data includes a structured data table, a streaming data table, and a file table;

[0219] The data source includes a structured database corresponding to the structured data table, a streaming data source corresponding to the streaming data table, and a file system data source corresponding to the file table.

[0220] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0221] Based on the same technical concept, the embodiment of the present application also provides a computer device. Figure 8 FIG. 1 is a schematic diagram of a computer device according to an embodiment of the present invention, comprising:

[0222] Processor 801, memory 802, and bus 803. Memory 802 stores machine-readable instructions executable by processor 801. Processor 801 is configured to execute the machine-readable instructions stored in memory 802. When the machine-readable instructions are executed by processor 801, processor 801 executes steps S101 to S105.

[0223] The above-mentioned memory 802 includes a memory 8021 and an external memory 8022; the memory 8021 here is also called an internal memory, which is used to temporarily store the calculation data in the processor 801, as well as the data exchanged with the external memory 8022 such as a hard disk. The processor 801 exchanges data with the external memory 8022 through the memory 8021. When the computer device is running, the processor 801 and the memory 802 communicate through the bus 803, so that the processor 801 executes the execution instructions mentioned in the above method embodiment.

[0224] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the question-and-answer method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0225] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the software update method described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0226] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0227] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0228] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0229] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0230] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0231] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0232] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A question-answering method, characterized in that: include: According to the data type of the batch data to be entered and the data acquisition method matching the data type, a database matching the data type is constructed, and data association relationships between the batch data are constructed, including: if the data type includes a structured data type, determining that the data acquisition method includes a data source connection method; connecting the data source of the structured batch data according to the data source connection method, acquiring the batch data from the connected data source, and extracting various data fields of the batch data; constructing a structured database corresponding to the structured data type based on the acquired batch data and the extracted data fields, and constructing data association relationships between the batch data based on correlations between the data fields of the batch data; In the case where the data type includes an unstructured data type, determining that the data acquisition method includes an interface upload method; according to the interface upload method, obtaining each data folder uploaded by the user and the application scenario of each data folder through the file upload interface; the data folder includes batch files; based on parallel parsing of the batch files in each data folder, determining the file vector of each unstructured file in the batch file, and constructing a file index corresponding to the batch file according to the file vector and the application scenario; constructing an unstructured database corresponding to the unstructured data type according to the file index and the file vector; constructing a data association relationship between the batch files according to the file vector, and constructing a data association relationship between the batch files and the batch data according to the file vector and the data field; wherein the data association relationship of any data in the batch data is used to indicate various other data that have storage association with the data; Performing intent classification and entity extraction on the target question input by the user to determine the query intent and query entity of the target question; Determining a target data type that matches the target question based on the query intent and the query entity; Acquire multiple pieces of target data matching the target question from the database according to the target data type and the data association relationship; An answer result matching the target question is determined based on the multiple pieces of target data.

2. The method according to claim 1, characterized in that The acquiring, from the database according to the target data type and the data association relationship, a plurality of target data matching the target question includes: screening, from a sub-database in the database that matches the target data type, a plurality of initial data matching the target question based on a data update time of each piece of database data in the sub-database, a first similarity between each piece of database data, and a second similarity between each piece of database data and the target question; Based on the data association relationship of each piece of the initial data, acquiring associated data related to the initial data; A third similarity between the associated data and the target question is determined, and based on the third similarity and the second similarity, a plurality of target data matching the target question is determined from the initial data and the associated data.

3. The method according to claim 1, characterized in that Determining, based on the plurality of target data, an answer result that matches the target question includes: generating at least one entity classification under a target query entity based on the plurality of target data and the query entity; the entity classification is used to be displayed to a user; the target query entity includes at least part of the query entity; Based on the target entity classification selected by the user under various target query entities, determining matching data matching the target entity classification from the target data; Based on the matching data, an answer result matching the target question is generated.

4. The method according to claim 1, wherein Determining, based on the plurality of target data, an answer result that matches the target question includes: If no answer can be determined based on the plurality of target data, determining whether the user authorizes the online search; If yes, obtaining network data matching the target question based on the multiple target data and the target question; An answer result matching the target question is generated based on the network data and the target data.

5. The method according to claim 4, characterized in that Generating an answer result matching the target question based on the network data and the target data includes: Filtering the network data according to the data update time of the target data and the data field or file index corresponding to the target data to obtain filtered network data; Storing the filtered network data in a database that matches the unstructured data type, and determining whether to use the unstructured data type as a new target data type based on the target data type; If so, the unstructured data type is used as a new target data type, and the step of obtaining multiple target data matching the target question from the database based on the target data type and the data association relationship is returned until an answer result matching the target question is obtained.

6. The method according to claim 1, characterized in that The structured batch data includes a structured data table, a streaming data table and a file table; The data source includes a structured database corresponding to the structured data table, a streaming data source corresponding to the streaming data table, and a file system data source corresponding to the file table.

7. A question-answering device, characterized in that: include: A construction module is configured to construct a database matching the data type of the batch data to be entered and a data acquisition method matching the data type, and to establish data association relationships between the batch data, including: if the data type includes a structured data type, determining that the data acquisition method includes a data source connection method; connecting the data source of the structured batch data according to the data source connection method, acquiring the batch data from the connected data source, and extracting various data fields of the batch data; constructing a structured database corresponding to the structured data type based on the acquired batch data and the extracted data fields, and establishing data association relationships between the batch data based on correlations between the data fields of the batch data; In the case where the data type includes an unstructured data type, determining that the data acquisition method includes an interface upload method; according to the interface upload method, obtaining each data folder uploaded by the user and the application scenario of each data folder through the file upload interface; the data folder includes batch files; based on parallel parsing of the batch files in each data folder, determining the file vector of each unstructured file in the batch file, and constructing a file index corresponding to the batch file according to the file vector and the application scenario; constructing an unstructured database corresponding to the unstructured data type according to the file index and the file vector; constructing a data association relationship between the batch files according to the file vector, and constructing a data association relationship between the batch files and the batch data according to the file vector and the data field; wherein the data association relationship of any data in the batch data is used to indicate various other data that have storage association with the data; The first determination module is used to perform intent classification and entity extraction on the target question input by the user, and determine the query intent and query entity of the target question; A second determination module is used to determine a target data type that matches the target question according to the query intent and the query entity; An acquisition module, configured to acquire, from the database, a plurality of target data matching the target question according to the target data type and the data association relationship; The third determination module is used to determine an answer result that matches the target question based on the multiple target data.

8. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor performs the steps of the question-answering method according to any one of claims 1 to 6.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a computer device, the computer device performs the steps of the question-answering method according to any one of claims 1 to 6.

Images