Question and answer method, device and electronic equipment
By building a document slice library in the government system and using the feature value matching degree of document slices to automatically find answers, the problem of low efficiency of manual customer service queries is solved, and fast and accurate answers are provided while reducing costs are achieved.
Patent Information
- Application Number
- CN202110529809.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-14
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2041-05-14
AI Technical Summary
The current method of obtaining answers through manual customer service in government systems is inefficient and costly.
By employing document slicing library technology, the answer slice is automatically determined from the document slicing library by obtaining the target question and utilizing the feature value matching degree of the document slices, thereby improving query efficiency.
It enables the rapid and accurate provision of answers within the government system, reduces question-and-answer costs, avoids redundant information in answers, and improves the accuracy of answers.
Smart Images

Figure CN115344674B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and more particularly to a question-and-answer method, apparatus, and electronic device. Background Technology
[0002] In the realm of government services, users can ask questions about issues they care about on the government service system, and the system will provide corresponding answers. For example, if a user asks about the "processing procedure for service A" on the government service system, the system can display the specific procedures and precautions for processing service A on the user's page, thereby improving the efficiency of processing various services for users.
[0003] Currently, user questions are answered through human customer service. For example, customer service representatives in government service systems search for answers to user questions in a pre-set database and then display the answers to users through the government service system.
[0004] As can be seen from the above, obtaining answers through human customer service is inefficient and costly. Summary of the Invention
[0005] This application provides a question-and-answer method, apparatus, and electronic device to improve the efficiency of answer retrieval.
[0006] Firstly, embodiments of this application provide a question-and-answer method, including:
[0007] Identify the target problem;
[0008] In the constructed document slice library, based on the feature values of each of the N first document slices, a first matching degree between each first document slice and the target problem is determined, wherein the document slice library includes the N first document slices and the feature values of each first document slice, and N is a positive integer;
[0009] Based on the first matching degree between each first document slice and the target question, the answer slice for the target question is determined from the N first document slices.
[0010] In some embodiments, determining the first feature value of each word in the target question based on the first feature value of each word in the first document slice includes:
[0011] For each word in the target problem, if the first document slice contains the word, then the first feature value of the word in the first document slice is obtained from the document slice library;
[0012] If the first document slice does not contain the word, then the first feature value of the word in the first document slice is determined to be a first numerical value.
[0013] In some embodiments, determining a second matching degree between the first document slice and the target question based on a first feature value of each word in the target question within the first document slice includes:
[0014] Based on the first feature value of each word in the target question in the first document slice, the second matching degree between the first document slice and the target question is determined using the BM25 matching method.
[0015] In some embodiments, determining the feature value corresponding to the first document slice based on the occurrence frequency and inverse text frequency of each word in the first document slice includes:
[0016] The product of the frequency of occurrence of each word in the first document slice and the inverse text frequency is determined as the feature value corresponding to the first document slice.
[0017] In some embodiments, constructing the document slice library based on the feature values corresponding to each first document slice includes:
[0018] Based on the feature value corresponding to each first document slice, the first document slice is inverted and the index of each first document slice is determined.
[0019] The document slice library is constructed based on each of the indexed first document slices.
[0020] In some embodiments, determining the answer slice for the target question from the N first document slices based on a first matching degree between each first document slice and the target question includes:
[0021] The first document slice with the highest first matching degree with the target question among the N first document slices is determined as the answer slice for the target question.
[0022] In some embodiments, the method further includes:
[0023] This displays a slice of the answer to the target question.
[0024] In some embodiments, determining the feature value corresponding to the first document slice includes:
[0025] Determine the frequency of occurrence of each word in the first document slice;
[0026] Determine the inverse text frequency of each word in the first document slice;
[0027] The feature value corresponding to the first document slice is determined based on the occurrence frequency and inverse text frequency of each word in the first document slice.
[0028] Secondly, embodiments of this application provide a question-and-answer device, including:
[0029] The acquisition unit is used to acquire the target problem;
[0030] The first determining unit is configured to determine a first matching degree between each first document slice and the target question based on the feature value of each first document slice in the constructed document slice library, wherein the document slice library includes the N first document slices and each first document, and is configured to determine the answer slice of the target question from the N first document slices based on the first matching degree between each first document slice and the target question.
[0031] In some embodiments, the first determining unit is specifically configured to, for each of the N first document slices, determine a second matching degree between the first document slice and the target question based on the feature values of the first document slice; determine at least one of a third matching degree between the title in the first document slice and the target question, a fourth matching degree between the body text in the first document slice and the target question, and a fifth matching degree between a sentence in the body text of the first document slice and the target question; and determine a first matching degree between the first document slice and the target question based on the second matching degree and at least one of the third, fourth, and fifth matching degrees.
[0032] In some embodiments, the first determining unit is specifically configured to determine the matching degree between each sentence in the text of the first document slice and the target question; and to determine the maximum matching degree among the matching degrees between each sentence in the text of the first document slice and the target question as the fifth matching degree.
[0033] In some embodiments, the first determining unit is specifically configured to determine the sum of the second matching degree, the third matching degree, the fourth matching degree, and the fifth matching degree as the first matching degree between the first document slice and the target question.
[0034] In some embodiments, the feature values of the first document slice include a first feature value of each word in the first document slice. The first determining unit is specifically configured to determine the first feature value of each word in the target question in the first document slice based on the first feature value of each word in the first document slice; and to determine a second matching degree between the first document slice and the target question based on the first feature value of each word in the target question in the first document slice.
[0035] In some embodiments, the first determining unit is specifically configured to, for each word in the target question, if the first document slice contains the word, obtain a first feature value of the word in the first document slice from the document slice library; if the first document slice does not contain the word, determine that the first feature value of the word in the first document slice is a first numerical value.
[0036] In some embodiments, the first determining unit is specifically configured to determine a second matching degree between the first document slice and the target question using the BM25 matching method based on the first feature value of each word in the target question in the first document slice.
[0037] In some embodiments, the apparatus further includes a construction unit, which is configured to acquire a target type document set, the target type document set including multiple target type documents; slice each target type document according to the characteristics of the target type documents to obtain at least one second document slice for each target type document; for each second document slice, combine the second document slice with the title to which the second document slice belongs to form a first document slice, and determine the feature value corresponding to the first document slice; and construct the document slice library according to the feature value corresponding to each first document slice.
[0038] In some embodiments, the construction unit is specifically configured to: identify the title of the target type document based on the characteristics of the target type document to obtain the title information of the target type document; and slice the target type document based on the title information of the target type document to obtain at least one second document slice of the target type document.
[0039] In some embodiments, the construction unit is specifically configured to: if there are no second-level headings under the first-level headings of the target type document, then treat the body text under the first-level headings as a second document slice; if there are second-level headings under the first-level headings of the target type document and there is body text under the second-level headings, then treat the body text under the second-level headings as a second document slice; if there are second-level headings under the first-level headings of the target type document and there is no body text between adjacent second-level headings, then treat the adjacent second-level headings as a second document slice.
[0040] In some embodiments, the construction unit is specifically configured to determine the occurrence frequency of each word in the first document slice; determine the inverse text frequency of each word in the first document slice; and determine the feature value corresponding to the first document slice based on the occurrence frequency and inverse text frequency of each word in the first document slice.
[0041] In some embodiments, the construction unit is specifically used to determine the product of the occurrence frequency and inverse text frequency of each word in the first document slice as the feature value corresponding to the first document slice.
[0042] In some embodiments, the construction unit is specifically configured to perform inverted sorting of each first document slice according to the feature value corresponding to each first document slice, and determine the index of each first document slice; and construct the document slice library according to each first document slice with an index.
[0043] In some embodiments, the second determining unit is specifically used to determine the first document slice with the largest first matching degree with the target question among the N first document slices as the answer slice for the target question.
[0044] In some embodiments, the apparatus further includes a display unit for displaying slices of the answer to the target question.
[0045] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory;
[0046] The memory is used to store computer programs;
[0047] The processor is configured to execute the computer program to implement the method described in the first aspect above.
[0048] Fourthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect.
[0049] Fifthly, embodiments of this application provide a computer program product comprising a computer program stored in a readable storage medium, wherein at least one processor of a computer can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the computer to perform the method described in the first aspect.
[0050] The question-answering method, apparatus, and electronic device provided in this application obtain a target question; in a pre-constructed document slice library, based on the feature values of each of the N first document slices, determine a first matching degree between each first document slice and the target question; and based on the first matching degree between each first document slice and the target question, determine the answer slice for the target question from the N first document slices. This application uses document slices as the smallest granularity of the answer, automatically querying the answer slice for the target question in the document slice library, thus improving question-answering efficiency and reducing question-answering costs. Attached Figure Description
[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0052] Figure 1 This is a schematic diagram of a system according to an embodiment of this application;
[0053] Figure 2 This is a schematic diagram of a distributed system according to an embodiment of this application;
[0054] Figure 3 This is a schematic diagram illustrating an application scenario according to an embodiment of this application;
[0055] Figure 4 This is a schematic diagram illustrating another application scenario provided by an embodiment of this application;
[0056] Figure 5 This is a schematic diagram illustrating another application scenario involved in the embodiments of this application;
[0057] Figure 6 A flowchart illustrating a question-and-answer method provided in an embodiment of this application;
[0058] Figure 7 This is a schematic diagram of a slice involved in an embodiment of this application;
[0059] Figure 8 A flowchart illustrating a question-and-answer method provided in an embodiment of this application;
[0060] Figure 9 A schematic diagram of the structure of the question-and-answer device provided in the embodiments of this application;
[0061] Figure 10 This is a block diagram of an electronic device involved in an embodiment of this application. Detailed Implementation
[0062] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0063] It should be understood that, in the embodiments of the present invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.
[0064] In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0065] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.
[0066] The embodiments of this application are applied to the field of software testing technology, and in particular to the legality check of requirement data, so as to generate test cases stably and efficiently based on the legal requirement data.
[0067] To facilitate understanding of the embodiments of this application, the relevant concepts involved in the embodiments of this application will be briefly introduced as follows:
[0068] Unstructured text: This refers to data with irregular or incomplete data structures, no predefined data model, and data that is inconvenient to represent using a two-dimensional logical table in a database.
[0069] Elastic Search: A search server based on Lucene that provides a distributed, real-time analytics search engine.
[0070] Recall: Trigger as many correct results as possible from the full set of information.
[0071] Cold start: This is a stage when a product (or new feature) is first launched, facing difficulties such as verifying market demand, lack of data, and lack of users.
[0072] Term frequency (TF) features are features derived from the frequency of a word in the current document.
[0073] Inverse Document Frequency (IDF) is the reciprocal of the number of times a word appears in all documents. If a word appears frequently in all documents, then the word has little information. IDF can be used to correct word frequency metrics.
[0074] An Application Programming Interface (API) is a set of predefined functions, or an agreement between different components of a software system. Its purpose is to provide applications and developers with the ability to access a set of routines based on certain software or hardware, without needing to access the source code or understand the details of the internal workings.
[0075] Figure 1 This is a schematic diagram of a system according to an embodiment of the present application, including a terminal device 10 and a server 20.
[0076] In some embodiments, server 20 may be a standalone physical server.
[0077] In some embodiments, server 20 may be a server cluster or a distributed system consisting of multiple physical servers. Taking a blockchain as an example of a distributed system... Figure 2As shown, the blockchain 100 includes multiple nodes 101, and the server 20 can be one of the nodes 101. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer. The blockchain underlying platform can include processing modules such as user management, basic services, smart contracts, and operational monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between users' real identities and blockchain addresses (access management). Under authorization, it also monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic service module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic service first performs interface adaptation parsing and authentication (interface adaptation), and then encrypts the business information using a consensus algorithm (consensus management). The blockchain process involves several layers of data transmission and processing. The first layer, the blockchain service layer, provides basic capabilities and implementation frameworks for typical applications. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). The second layer provides blockchain-based application services for business stakeholders. The third layer provides application services based on blockchain solutions for use by business participants.
[0078] In some embodiments, server 20 may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
[0079] Cloud services, also known as artificial intelligence cloud services, are generally referred to as AIaaS (AI as a Service). This is currently a mainstream service model for artificial intelligence platforms. Specifically, AIaaS platforms break down several common AI services and provide them as independent or packaged services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI framework and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services.
[0080] A cloud database can be understood as a database located in the cloud. A database can be viewed as an electronic filing cabinet, a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of applications.
[0081] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as SQL (Structured Query Language) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages.
[0082] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.
[0083] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.
[0084] Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.
[0085] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network to work together through application software or application interfaces to provide data storage and business access functions to the outside world.
[0086] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.
[0087] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.
[0088] In some embodiments, the terminal device 10 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these.
[0089] Terminal device 10 and server 20 can be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions on this.
[0090] In some embodiments of this application, the server is further configured to create the aforementioned document slice library, that is, the server slices the target type document according to the characteristics of the target type document, and constructs the document slice library based on the sliced document slices.
[0091] In some embodiments, other electronic devices besides the server may execute the method of the embodiments of this application to create the above-mentioned document slice library, and then store the created document slice library on the server.
[0092] like Figure 1 As shown, in this embodiment of the application, the server 20 can access a built document slice library, for example, the server 20 stores a built document slice library, or the server 20 accesses a document slice library stored on other devices.
[0093] In this embodiment, the user sends a target question through the terminal device 10. The server 20 queries the document slice library for the answer slice that matches the target question based on the target question sent by the terminal device 10, and sends the answer slice that matches the target question to the terminal device 10, so that the terminal device 10 presents the answer slice to the user.
[0094] The document slice library in this application includes document slices of target type documents. The answer slices for the target question are automatically queried in the document slice library, which improves the efficiency of question answering and reduces the cost of question answering.
[0095] In addition, the embodiments of this application perform answer matching at the document slice granularity, which reduces the amount of matching data, avoids unnecessary redundant information in the answers, and improves the accuracy of the answers.
[0096] Figure 3This is a schematic diagram illustrating an application scenario according to an embodiment of this application, such as... Figure 3 As shown, the question-and-answer method provided in this application embodiment can be provided to enterprise customers as a cloud service to help provide basic question-and-answer services and retrieve relevant slices from massive documents and return them to users.
[0097] like Figure 3 As shown, the cloud server 31 stores a document slice library, which includes multiple first document slices. The terminal device 32 is connected to the cloud server 31 via a network, which can be an intranet, the Internet, Global System for Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, voice communication network, or other wireless or wired networks.
[0098] In some embodiments, cloud server 31 provides an application interface for terminal device 32 to use. For example, cloud server 31 provides an API for data augmentation services.
[0099] like Figure 3 As shown, the user inputs the target question on the terminal device 32, and the terminal device 32 processes the target question to obtain the input parameters shown in Table 1 below:
[0100] Table 1
[0101]
[0102] Terminal device 32 sends the above input parameters to cloud server 31. Cloud server 31 searches for the answer slice in the document slice library based on the input parameters. Cloud server 31 processes the answer slice to obtain the output parameters shown in Table 2.
[0103] Table 2
[0104]
[0105] The descriptions of the elements in the ResponceList result unit in Table 2 are shown in Table 3:
[0106] Table 3
[0107] DocumentID String Document ID containing the answer PieceID String Answer fragment ID Title String Document title Heading_Level_1 String The first-level heading where the clip is located Heading_Level_2 String The second-level heading where the excerpt is located Answer String The main text of the answer fragment
[0108] After receiving the output parameters sent by the cloud server 31, the terminal device 32 presents a slice of the answer to the target question to the user based on the output parameters, thereby achieving a fast and accurate determination of the answer to the target question.
[0109] Figure 4 This is a schematic diagram of another application scenario provided by an embodiment of this application. This application scenario can be understood as a human customer service scenario.
[0110] like Figure 4 As shown, it includes: terminal device 41, terminal device 42, and server 43. Terminal device 41 and terminal device 42 are connected via a network, and terminal device 42 and server 43 are connected via a network.
[0111] Among them, terminal device 41 is a user-side terminal device, on which the user inputs the target question.
[0112] Terminal device 42 is the terminal device on the human customer service side. Terminal device 41 sends the target question entered by the user to terminal device 42, and terminal device 42 presents the target question to the human customer service representative. The human customer service representative sends the target question to server 43 through terminal device 42. Server 43 searches for the corresponding answer slice in the document slice library and sends the matching answer slice to terminal device 42. The human customer service representative further edits the answer slice sent by server 43 and directly replies to the user.
[0113] For example, such as Figure 5 As shown, the user's target question is "For a user in state 1, what review materials are needed to process business A?" The customer service representative uses the intelligent customer service assistant to search the document slice library and finds three answer slices corresponding to the target question, namely: Answer slice 1 "(I) Meeting condition 1, 1. Material 1, 2. Material 2, 3. Material 3", Answer slice 2 "(III) For users in state 2, 1. Material 1, etc.", Answer slice 3 "II. Processing method and procedure, online processing: apply for retrieval based on condition (I) (V)".
[0114] Based on the three answer slices mentioned above, the customer service representative edits and outputs the answer to the target question. For example, answer slice 1 is output to the user as the answer to the target question.
[0115] It should be noted that, Figure 3 or Figure 4 An exemplary illustration of an application scenario of this application is shown. The application scenarios of this application include, but are not limited to, those described above. Figure 3 or Figure 4 As shown.
[0116] The technical solutions of the embodiments of this application will be described in detail below through some examples. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0117] Example 1
[0118] First, the creation process of the document slice library involved in the embodiments of this application will be introduced.
[0119] Figure 6 This is a flowchart illustrating a question-and-answer method provided in an embodiment of this application, as shown below. Figure 6 As shown, the method in this application embodiment includes:
[0120] S601. Obtain the target type document set, which includes multiple target type documents.
[0121] The execution subject of this application embodiment is a device for querying the answer slice corresponding to a target question in a document slice library. This device may include... Figure 1 The server shown.
[0122] In some embodiments, the target type document mentioned above is a government document.
[0123] The target type document set includes multiple target type documents, such as multiple government documents.
[0124] S602. Based on the characteristics of the target type document, slice each target type document to obtain at least one second document slice for each target type document.
[0125] The target type documents in this application embodiment are relatively standardized, typically including heading information such as first-level headings and second-level headings, as well as body text. Based on these characteristics of the target type documents, each target type document in the target type document set is sliced to obtain at least one second document slice for each target type document.
[0126] In some embodiments, S602 includes the following steps S602-A1 and S602-A2:
[0127] S602-A1. Based on the characteristics of the target type document, identify the title of the target type document to obtain the title information of the target type document;
[0128] S602-A2. Based on the title information of the target type document, slice the target type document to obtain at least one second document slice of the target type document.
[0129] Because the target document type is relatively standardized, pattern matching can be used to identify the first-level and second-level headings, thereby revealing the document's structure and automatically segmenting it into non-fixed-length slices. Specifically, based on the characteristics of the target document, headings are identified to obtain heading information; based on this heading information, the document is sliced to obtain at least one second document slice.
[0130] For example, the initial identification of heading information in the target type document, such as the preliminary identification of two-level headings in the target type document:
[0131] 1. First-level headings mainly fall into two categories:
[0132] 1) Text that starts with "Chapter *", is limited to a preset number of characters (e.g., 30), and is in a separate paragraph, such as "Chapter 1 Subsidy Policy".
[0133] 2) Text that begins with Chinese Arabic numerals, is limited to a preset number of characters (e.g., 30), and is in a separate paragraph, such as "I. Application Requirements" or "II. Online Application Methods and Procedures".
[0134] 2. Second-level headings generally fall into two categories:
[0135] 1) Text that begins with “Section *” or “Article *”, is limited to a preset number of characters (e.g., 30), and is in a separate paragraph, such as “Article 3, Subsidies for Retired Personnel”.
[0136] 2) Text that begins with parentheses, contains Chinese characters and Arabic numerals, and is limited to a preset number of characters (e.g., 30), and is in a separate paragraph, such as "(I) Retiree Subsidy Materials".
[0137] Based on the above rules, the title information of the target type document is identified, and then S602-A2 is executed to slice the target type document based on the title information.
[0138] In some embodiments, the methods for slicing the target type document based on the title information of the target type document in S602-A2 above include, but are not limited to, the following:
[0139] Case 1: If there are no second-level headings under the first-level headings of the target type document, then the body text under the first-level headings will be treated as a second document slice.
[0140] Case 2: If a target document has a second-level heading under a first-level heading and body text under the second-level heading, then the body text under the second-level heading is treated as a second document slice.
[0141] Case 3: If a first-level heading in the target type document contains a second-level heading and there is no body text between adjacent second-level headings, then the adjacent second-level headings are treated as a second document slice.
[0142] For example, such as Figure 7 As shown, the guide for applying for A includes three first-level headings: I. Application Requirements; II. Application Methods and Procedures; and III. Application Materials. The first-level heading "Application Requirements" includes multiple second-level headings. The first-level heading "Application Methods and Procedures" does not include second-level headings but includes the main text. The first-level heading "Application Materials" includes second-level headings, and the second-level headings include the main text.
[0143] like Figure 7 As shown, if the first-level heading "Processing Methods and Procedures" does not include second-level headings, then the main text under "Processing Methods and Procedures" is divided into a second document slice, such as second document slice 2. If the first-level heading "Application Materials" includes second-level headings, and each second-level heading includes main text, then the main text under each second-level heading under "Application Materials" is divided into a second document slice, such as second document slice 3. If the first-level heading "Application Conditions" includes multiple second-level headings, and there is no main text between adjacent second-level headings, then adjacent second-level headings are treated as a single second document slice, such as second document slice 1.
[0144] It should be noted that the above is one method of slicing a target type document based on the title information of the target type document in the embodiments of this application. The slicing methods in the embodiments of this application include, but are not limited to, the above-mentioned methods.
[0145] S603. For each second document slice, combine the second document slice with the title to which the second document slice belongs to form a first document slice, and determine the feature value corresponding to the first document slice.
[0146] After slicing the target type document according to S602 above, the slices are stored in the database.
[0147] Specifically, the second document slice is combined with the title to which the second document slice belongs to form the first document slice, for example... Figure 7As shown, taking the second document slice 2 as an example, the second document slice 2 has a first-level heading "Processing Methods and Procedures" but no second-level headings. Therefore, the second document slice 2 and the first-level heading "Processing Methods and Procedures" are considered as a first document slice. Taking the second document slice 3 as an example, the second document slice 3 has a first-level heading "Application Materials" and a second-level heading "(I) Meeting Condition 1". The first-level heading and the second-level heading of the second document slice 2 are considered as a first document slice.
[0148] As can be seen from the above, one second document slice corresponds to one first document slice.
[0149] In some embodiments, the ElasticSearch tool is used to automatically build the index of each first document slice and extract the features of the first document slice.
[0150] In some embodiments, the present application uses the term frequency (TF) and inverse document frequency (IDF) features of the first document slice to characterize the features of the first document slice.
[0151] In some embodiments, determining the feature value corresponding to the first document slice in S603 above includes the following steps S603-A1 to S603-A3:
[0152] S603-A1. Determine the frequency of occurrence of each word in the first document slice, wherein the frequency of occurrence of each word in the first document slice can be understood as the word frequency feature of the first document slice.
[0153] In one possible implementation, taking a first document slice as an example, the frequency of each word in the first document slice is determined according to the following formula (1):
[0154]
[0155] Where w is a word in the first document slice, ND(w) represents the number of words containing w in the first document slice, and N w TF(w) represents the total number of words in the first document slice, and TF(w) represents the frequency of w in the first document slice.
[0156] It should be noted that the above formula (1) is only an example. Any transformation of the above formula (1) also falls within the protection scope of the embodiments of this application. For example, equivalent transformations of the above formula (1), or multiplying, dividing, adding or subtracting one or more parameters of the above formula (1) all fall within the protection scope of the embodiments of this application.
[0157] S603 - A2. Determine the inverse document frequency of each word in the first document slice, where the inverse document frequency of each word in the first document slice can be understood as the inverse document frequency feature of the first document slice.
[0158] In a possible implementation, determine the inverse document frequency of each word in the first document slice according to the following formula (2):
[0159]
[0160] where, N d represents the total number of documents in the preset corpus, and N(w) represents the number of documents in the preset corpus that contain the word w.
[0161] It should be noted that the above formula (2) is only an example, and any deformation of the above formula (2) also belongs to the protection scope of the embodiments of this application. For example, equivalent deformations of the above formula (2), or multiplying, dividing, adding or subtracting one or several parameters to the above formula (2), etc., all belong to the protection scope of the embodiments of this application.
[0162] S603 - A3. Determine the eigenvalue corresponding to the first document slice according to the occurrence frequency and inverse document frequency of each word in the first document slice.
[0163] In a possible implementation, determine the occurrence frequency and inverse document frequency of each word in the first document slice as the eigenvalue corresponding to the first document slice.
[0164] In a possible implementation, determine the product of the occurrence frequency and inverse document frequency of each word in the first document slice as the eigenvalue corresponding to the first document slice.
[0165] Exemplarily, determine the eigenvalue corresponding to the first document slice according to the following formula (3):
[0166] TF - IDF(w) = TF(w) * IDT(w) (3)
[0167] TF - IDF is a statistical method used to evaluate the importance of a word for a document set or a single document in a corpus. The importance of a word increases proportionally with the number of times it appears in the document, but at the same time decreases inversely with the frequency of its appearance in the corpus. Various forms of TF - IDF weighting are often used by search engines as a measure or rating of the relevance between a document and a user query. For example, "de" appears in almost all documents, so it has less information content than "document Q&A". Therefore, the inverse document frequency can well correct the word frequency index. Therefore, by constructing the TF - IDF features of each word in the document, the feature representation of the document can be obtained.
[0168] In one possible implementation, the occurrence frequency TF(w) and inverse text frequency IDF(w) of each word in the first document slice, as well as the product of the occurrence frequency and inverse text frequency of each word in the first document slice, TF-IDF(w), are determined as the feature values corresponding to the first document slice.
[0169] S604. Construct a document slice library based on the feature values corresponding to each first document slice.
[0170] In this embodiment of the application, each first document slice is sorted according to the feature value corresponding to each first document slice, and the index of each first document slice is determined. These sorted first document slices constitute a document slice library.
[0171] In some embodiments, each first document slice can be sorted sequentially according to the feature value corresponding to each first document slice, and the index of each first document slice can be determined. These indexed first document slices constitute a document slice library.
[0172] In some embodiments, each first document slice can be inverted according to the feature value corresponding to each first document slice, and the index of each first document slice can be determined. A document slice library can be constructed based on each indexed first document slice.
[0173] This step sorts each first document slice according to its corresponding feature value and then stores it in the database, making it easier to search within the document slice database later.
[0174] This application embodiment automatically identifies the title and body information of the target type document by leveraging its features, and obtains slices of different lengths based on this information, avoiding the segmentation of semantic units caused by a fixed window. Furthermore, considering the often long length of answers in the government affairs field, this application embodiment uses document slices as the smallest granularity of the answer, thereby improving the accuracy of the answer.
[0175] The above section provides a detailed introduction to the creation process of a document slice library. Building on this foundation, the following section describes the process of searching for answer slices for a target question within the created document slice library.
[0176] Figure 8 This is a flowchart illustrating a question-and-answer method provided in an embodiment of this application, as shown below. Figure 8 As shown, it includes:
[0177] S801, Obtaining the target problem.
[0178] Taking the government affairs sector as an example, users can open the government affairs system on their terminal devices, enter the target question in the corresponding question selection box of the government affairs system, and search for the answer to the target question.
[0179] In such Figure 3 In the application scenario shown, the target problem is that the user sends the information directly to the cloud server through the terminal device.
[0180] In such Figure 4 In the application scenario shown, the target question is sent by the user to the corresponding terminal device 42 of the human customer service via terminal device 41, and then the terminal device 42 sends the target question to the server 43.
[0181] S802. In the constructed document slice library, based on the feature values of each of the N first document slices, determine the first matching degree between each first document slice and the target problem.
[0182] The document slice library includes N first document slices and feature values for each first document slice, where N is a positive integer.
[0183] In S802 above, the methods for determining the first matching degree between each first document slice and the target problem based on the feature values of each of the N first document slices include, but are not limited to, the following:
[0184] Method 1: For each of the N first document slices, determine the second matching degree between the first document slice and the target question based on the feature value of the first document slice, and then determine the second matching degree as the first matching degree between the first document slice and the target question.
[0185] Method 2, the above S802 includes the following S802-A1 to S802-A3:
[0186] S802-A1. For each of the N first document slices, determine the second matching degree between the first document slice and the target problem based on the feature values of the first document slice.
[0187] S802-A2, determine at least one of the following: the third degree of matching between the title and the target question in the first document slice; the fourth degree of matching between the body text and the target question in the first document slice; and the fifth degree of matching between the sentence in the body text and the target question in the first document slice.
[0188] S802-A3. Determine the first matching degree between the first document slice and the target question based on the second matching degree and at least one of the third, fourth and fifth matching degrees.
[0189] The process of determining the second matching degree between the first document slice and the target problem based on the feature values of the first document slice in S802-A1 above will be described below.
[0190] In some embodiments, if the feature values of the first document slice include the first feature value of each word in the first document slice, then the above S802-A1 includes S802-A11 and S802-A12:
[0191] S802-A11. Based on the first feature value of each word in the first document slice, determine the first feature value of each word in the target question in the first document slice;
[0192] S802-A12. Determine the second matching degree between the first document slice and the target question based on the first feature value of each word in the target question in the first document slice.
[0193] In one possible implementation, S802-A11 includes: for each word in the target problem, if the first document slice contains the word, then obtain the first feature value of the word in the first document slice from the document slice library; if the first document slice does not contain the word, then determine the first feature value of the word in the first document slice as a first numerical value. Optionally, the first numerical value is 0.
[0194] Next, based on the first feature value of each word in the target question in the first document slice, the second matching degree between the first document slice and the target question is determined.
[0195] In one example, the second degree of matching between the first document slice and the target question is determined based on the sum of the first feature values of each word in the target question in the first document slice. For example, the larger the sum of the first feature values of each word in the target question in the first document slice, the larger the second degree of matching between the first document slice and the target question is determined. Conversely, the smaller the sum of the first feature values of each word in the target question in the first document slice, the smaller the second degree of matching between the first document slice and the target question is determined.
[0196] In another example, the second degree of matching between the first document slice and the target question is determined using the BM25 matching method, based on the first feature value of each word in the target question in the first document slice.
[0197] In some embodiments, the second matching degree between the first document slice and the target question is determined according to the following formula (4):
[0198]
[0199] Where k1 and b are adjustment factors, usually set empirically, typically k1 = 2 and b = 0.75; Q is the word set included in the target question, containing n words where n is a positive integer; qi is the i-th word in Q; d1 is the first document slice; d1l is the length of the first document slice d1; avgdl is the average length of all documents in the preset corpus; f1i is the frequency of qi in the first document slice d1; and IDF(qi)1 is the inverse document frequency of qi in the first document slice d1. Score(Q,d1) is the second matching degree between the first document slice and the target question.
[0200] In some embodiments, a third degree of matching between the title in the first document slice and the target question is determined based on a first feature value of each word in the target question in the title of the first document slice.
[0201] For example, the third degree of matching between the title and the target question in the first document slice is determined according to the following formula (5):
[0202]
[0203] Where d2 is the title of the first document slice, d2l is the length of the title of the first document slice, avgdl is the average length of all documents in the preset corpus, f2i is the frequency of qi in the title of the first document slice, and IDF(qi)2 is the inverse document frequency of qi in the title of the first document slice. Score(Q,d2) is the third degree of matching between the title of the first document slice and the target question.
[0204] In some embodiments, a fourth degree of matching between the text of the first document slice and the target question is determined based on a first feature value of each word in the target question in the text of the first document slice.
[0205] In some embodiments, the fourth matching degree between the text of the first document slice and the target question is determined according to the following formula (6):
[0206]
[0207] Where d3 is the text of the first document slice, d3l is the length of the text of the first document slice, avgdl is the average length of all documents in the preset corpus, f3i is the frequency of occurrence of qi in the text of the first document slice, and IDF(qi)3 is the inverse document frequency of qi in the text of the first document slice. Score(Q,d3) is the fourth matching degree between the text of the first document slice and the target question.
[0208] In some embodiments, determining the fifth matching degree between sentences in the text of the first document slice and the target question in S802-A2 above includes: determining the matching degree between each sentence in the text of the first document slice and the target question; and determining the maximum matching degree among the matching degrees between each sentence in the text of the first document slice and the target question as the fifth matching degree.
[0209] For example, the fifth degree of matching between a sentence in the first document slice and the target question is determined according to the following formula (7):
[0210]
[0211] Where d4 is a sentence in the first document slice, d4l is the length of the sentence, f4i is the frequency of qi in the sentence, and IDF(qi)4 is the inverse document frequency of qi in the sentence. Score(Q,d4) is the fifth degree of matching between the sentence and the target question.
[0212] According to the above formula (7), the matching degree between each sentence in the text of the first document slice and the target question can be determined. The maximum matching degree among the matching degrees between each sentence in the text of the first document slice and the target question is determined as the fifth matching degree.
[0213] In this second method, the first matching degree between the first document slice and the target question is determined based on the second matching degree and at least one of the third, fourth and fifth matching degrees, thereby improving the accuracy of determining the first matching degree between the first document slice and the target question.
[0214] In some embodiments, S802-A3 above includes: determining the sum of the second matching degree, the third matching degree, the fourth matching degree and the fifth matching degree as the first matching degree between the first document slice and the target question.
[0215] In some embodiments, S802-A3 includes: multiplying the second matching degree, the third matching degree, the fourth matching degree, and the fifth matching degree to determine the first matching degree between the first document slice and the target question.
[0216] In some embodiments, S802-A3 above includes: determining the average of the second matching degree, the third matching degree, the fourth matching degree, and the fifth matching degree as the first matching degree between the first document slice and the target question.
[0217] Optionally, the above averages are arithmetic averages.
[0218] Optionally, the above averages are weighted averages.
[0219] S803. Based on the first matching degree between each first document slice and the target question, determine the answer slice for the target question from the N first document slices.
[0220] In one possible implementation, the first document slice with the highest first matching degree with the target question among N first document slices is determined as the answer slice for the target question.
[0221] In one possible implementation, the first document slice among the N first document slices whose first matching degree with the target question is greater than a preset value is determined as the answer slice for the target question.
[0222] In one possible implementation, a predetermined number of first document slices with the highest first matching degree to the target question are selected from N first document slices and determined as the answer slices to the target question.
[0223] In some embodiments, this application further includes: displaying a slice of the answer to the target question.
[0224] The method of this application embodiment obtains the target question; in the constructed document slice library, according to the feature value of each of the N first document slices, determines the first matching degree between each first document slice and the target question, wherein the document slice library includes the N first document slices and the feature value of each first document slice, and N is a positive integer; according to the first matching degree between each first document slice and the target question, the answer slice of the target question is determined from the N first document slices. This method can automatically and cost-effectively build question-and-answer capabilities based on unstructured government documents, thereby reducing the time and manpower costs brought by manually writing question-and-answer pairs. In scenarios such as intelligent assistants for human customer service, relevant answer fragments are quickly recommended to human customer service, avoiding the need for human customer service to read a large number of policy documents. This method further improves the utilization rate of unstructured data in the government field based on traditional structured data automatic question-and-answer methods such as KBQA[6] and QA matching.
[0225] In addition, this application embodiment does not use a reading comprehension model to obtain the final answer. Instead, it constructs multi-granular features based on automatically identified information such as title, headings, and body text, sorts the recalled slices, and obtains the final answer slice. This reduces the amount of matching data, avoids unnecessary redundant information in the answer, and improves the accuracy of the answer.
[0226] Taking government documents as an example, this application uses government documents as its corpus. First, it automatically segments the documents according to their characteristics, constructing an answer fragment library. For user-submitted questions, it automatically recalls relevant fragments as candidate answers, and ranks the best answers based on multiple granularities and features, such as title, body text, and key sentences. Finally, it selects fragments from the provided document set as answers and provides the best-matching short answer. This question-answering scheme utilizes a large amount of easily accessible unstructured documents as corpus to automatically and quickly build government question-answering capabilities, reducing the cost of manually constructing question-answer pairs, solving the cold start problem caused by insufficient manually labeled data, and improving the efficiency of question-answering capability construction.
[0227] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.
[0228] It should also be understood that, in the various method embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0229] The above text combined Figures 6 to 8 The method embodiments of this application are described in detail below, in conjunction with... Figures 9 to 10 The following describes in detail the device embodiments of this application.
[0230] Figure 9 This is a schematic diagram of a question-and-answer device provided in an embodiment of this application. The question-and-answer device can be an electronic device or a component of an electronic device (e.g., an integrated circuit, a chip, etc.). The electronic device can be... Figure 1 The server shown.
[0231] like Figure 9 As shown, the question-and-answer device 300 may include: an acquisition unit 310, a first determination unit 320, and a second determination unit 330.
[0232] Acquisition unit 310 is used to acquire the target problem;
[0233] The first determining unit 320 is configured to determine a first matching degree between each first document slice and the target problem in the constructed document slice library based on the feature values of each first document slice among N first document slices, wherein the document slice library includes the N first document slices and the feature values of each first document slice, and N is a positive integer;
[0234] The second determining unit 330 is configured to determine the answer slice of the target question from the N first document slices based on the first matching degree between each first document slice and the target question.
[0235] In some embodiments, the first determining unit 320 is specifically configured to, for each of the N first document slices, determine a second matching degree between the first document slice and the target question based on the feature values of the first document slice; determine at least one of a third matching degree between the title in the first document slice and the target question, a fourth matching degree between the body text in the first document slice and the target question, and a fifth matching degree between a sentence in the body text of the first document slice and the target question; and determine a first matching degree between the first document slice and the target question based on the second matching degree and at least one of the third, fourth, and fifth matching degrees.
[0236] In some embodiments, the first determining unit 320 is specifically configured to determine the matching degree between each sentence in the text of the first document slice and the target question; and to determine the maximum matching degree among the matching degrees between each sentence in the text of the first document slice and the target question as the fifth matching degree.
[0237] In some embodiments, the first determining unit 320 is specifically configured to determine the sum of the second matching degree, the third matching degree, the fourth matching degree and the fifth matching degree as the first matching degree between the first document slice and the target question.
[0238] In some embodiments, the feature values of the first document slice include the first feature value of each word in the first document slice. The first determining unit 320 is specifically configured to determine the first feature value of each word in the target question in the first document slice based on the first feature value of each word in the first document slice; and to determine the second matching degree between the first document slice and the target question based on the first feature value of each word in the target question in the first document slice.
[0239] In some embodiments, the first determining unit 320 is specifically configured to, for each word in the target question, if the first document slice contains the word, obtain the first feature value of the word in the first document slice from the document slice library; if the first document slice does not contain the word, determine the first feature value of the word in the first document slice as a first numerical value.
[0240] In some embodiments, the first determining unit 320 is specifically configured to determine a second matching degree between the first document slice and the target question using the BM25 matching method based on the first feature value of each word in the target question in the first document slice.
[0241] In some embodiments, the apparatus further includes a construction unit 340, which is configured to acquire a target type document set, the target type document set including multiple target type documents; slice each target type document according to the characteristics of the target type documents to obtain at least one second document slice for each target type document; for each second document slice, combine the second document slice with the title to which the second document slice belongs to form a first document slice, and determine the feature value corresponding to the first document slice; and construct the document slice library according to the feature value corresponding to each first document slice.
[0242] In some embodiments, the construction unit 340 is specifically configured to: identify the title of the target type document based on the characteristics of the target type document to obtain the title information of the target type document; and slice the target type document based on the title information of the target type document to obtain at least one second document slice of the target type document.
[0243] In some embodiments, the construction unit 340 is specifically configured to: if there is no second-level heading under the first-level heading of the target type document, then treat the body text under the first-level heading as a second document slice; if there is a second-level heading under the first-level heading of the target type document and there is body text under the second-level heading, then treat the body text under the second-level heading as a second document slice; if there is a second-level heading under the first-level heading of the target type document and there is no body text between adjacent second-level headings, then treat the adjacent second-level headings as a second document slice.
[0244] In some embodiments, the construction unit 340 is specifically configured to determine the occurrence frequency of each word in the first document slice; determine the inverse text frequency of each word in the first document slice; and determine the feature value corresponding to the first document slice based on the occurrence frequency and inverse text frequency of each word in the first document slice.
[0245] In some embodiments, the construction unit 340 is specifically used to determine the product of the occurrence frequency and inverse text frequency of each word in the first document slice as the feature value corresponding to the first document slice.
[0246] In some embodiments, the construction unit 340 is specifically configured to perform inverted sorting of each first document slice according to the feature value corresponding to each first document slice, and determine the index of each first document slice; and construct the document slice library according to each first document slice with an index.
[0247] In some embodiments, the second determining unit 330 is specifically used to determine the first document slice with the largest first matching degree with the target question among the N first document slices as the answer slice for the target question.
[0248] In some embodiments, the device further includes a display unit 350 for displaying slices of the answer to the target question.
[0249] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be found in the method embodiments. To avoid repetition, further details are omitted here. Specifically, Figure 9 The apparatus shown can correspond to the respective subject in performing the method of the embodiments of this application, and the foregoing and other operations and / or functions of each module in the question-and-answer apparatus 100 are respectively for implementing Figures 6 to 8 For the sake of brevity, the corresponding processes in each method are not described in detail here.
[0250] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by the hardware processor, or executed by a combination of hardware and software modules in the processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0251] Figure 10 This is a block diagram of an electronic device according to an embodiment of this application. The device may be... Figure 1 The server shown is used to execute the methods described in the above embodiments, as detailed in the description of the above method embodiments.
[0252] Figure 10 The illustrated electronic device 200 includes a memory 201, a processor 202, and a communication interface 203. The memory 201, processor 202, and communication interface 203 are communicatively connected to each other. For example, the memory 201, processor 202, and communication interface 203 can be connected via a network. Alternatively, the electronic device 200 may also include a bus 204. The memory 201, processor 202, and communication interface 203 are communicatively connected to each other via the bus 204. Figure 10 It is an electronic device 200 in which the memory 201, processor 202, and communication interface 203 are connected to each other via bus 204.
[0253] The memory 201 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 201 can store programs, and when the program stored in the memory 201 is executed by the processor 202, the processor 202 and the communication interface 203 are used to execute the above methods.
[0254] The processor 202 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits.
[0255] The processor 202 can also be an integrated circuit chip with signal processing capabilities. In implementation, the method of this application can be completed through integrated logic circuits in the hardware of the processor 202 or through software instructions. The processor 202 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The software module can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 201, and the processor 202 reads information from memory 201 and, in conjunction with its hardware, completes the method of the embodiments of this application.
[0256] Communication interface 203 uses transceiver modules, such as, but not limited to, transceivers, to enable communication between electronic device 200 and other devices or communication networks. For example, data sets can be acquired through communication interface 203.
[0257] When the aforementioned electronic device 200 includes a bus 204, the bus 204 may include a path for transmitting information between various components of the electronic device 200 (e.g., memory 201, processor 202, communication interface 203).
[0258] According to this application, a computer storage medium is also provided, on which a computer program is stored, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions, which, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.
[0259] According to this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in the above-described method embodiments.
[0260] In other words, when implemented using software, it can be implemented wholly or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0261] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0262] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0263] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0264] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Furthermore, reference can be made between the various method embodiments and between the various device embodiments; identical or corresponding content in different embodiments can be mutually referenced, without further elaboration.
Claims
1. A question-and-answer method, characterized in that, include: Identify the target problem; In the constructed document slice library, based on the feature values of each of the N first document slices, a first matching degree between each first document slice and the target problem is determined, wherein the document slice library includes the N first document slices and the feature values of each first document slice, and N is a positive integer; Based on the first matching degree between each first document slice and the target question, the answer slice for the target question is determined from the N first document slices; The method further includes: acquiring a target type document set, and slicing each target type document according to the characteristics of the target type documents to obtain at least one second document slice for each target type document, wherein the target type document set includes multiple target type documents; For each second document slice, the second document slice and its corresponding title are combined to form a first document slice, and the feature value corresponding to the first document slice is determined. Then, the document slice library is constructed based on the feature value corresponding to each first document slice.
2. The method according to claim 1, characterized in that, The step of determining the first matching degree between each first document slice and the target question based on the feature values of N first document slices includes: For each of the N first document slices, a second matching degree between the first document slice and the target question is determined based on the feature values of the first document slice. Determine at least one of the following matching degrees: a third matching degree between the title in the first document slice and the target question, a fourth matching degree between the body text in the first document slice and the target question, and a fifth matching degree between a sentence in the body text of the first document slice and the target question; A first matching degree between the first document slice and the target question is determined based on the second matching degree and at least one of the third, fourth and fifth matching degrees.
3. The method according to claim 2, characterized in that, Determining the fifth matching degree between sentences in the text of the first document slice and the target question includes: Determine the matching degree between each sentence in the text of the first document slice and the target question; The maximum matching degree among the matching degrees between each sentence in the main text of the first document slice and the target question is determined as the fifth matching degree.
4. The method according to claim 2, characterized in that, Determining the first matching degree between the first document slice and the target question based on the second matching degree and at least one of the third, fourth, and fifth matching degrees includes: The sum of the second matching degree, the third matching degree, the fourth matching degree, and the fifth matching degree is determined as the first matching degree between the first document slice and the target question.
5. The method according to claim 2, characterized in that, The feature values of the first document slice include the first feature value of each word in the first document slice. The step of determining the second matching degree between the first document slice and the target question based on the feature values of the first document slice includes: Based on the first feature value of each word in the first document slice, determine the first feature value of each word in the target question in the first document slice; A second matching degree between the first document slice and the target question is determined based on the first feature value of each word in the target question in the first document slice.
6. The method according to claim 1, characterized in that, The step of slicing each target type document according to its characteristics to obtain at least one second document slice for each target type document includes: Based on the characteristics of the target type document, the title of the target type document is identified to obtain the title information of the target type document; Based on the title information of the target type document, the target type document is sliced to obtain at least one second document slice of the target type document.
7. The method according to claim 6, characterized in that, The step of slicing the target type document according to the title information of the target type document to obtain at least one second document slice of the target type document includes: If there are no second-level headings under the first-level headings of the target type document, then the body text under the first-level headings is treated as a second document slice. If a first-level heading of the target type document contains a second-level heading and the second-level heading contains body text, then the body text under the second-level heading is taken as a second document slice. If a first-level heading in the target type document contains a second-level heading and there is no body text between adjacent second-level headings, then the adjacent second-level headings are treated as a second document slice.
8. A question-and-answer device, characterized in that, include: The acquisition unit is used to acquire the target problem; The first determining unit is configured to determine a first matching degree between each first document slice and the target problem based on the feature value of each first document slice in the constructed document slice library, wherein the document slice library includes the N first document slices and the feature value of each first document slice, and N is a positive integer; The second determining unit is configured to determine the answer slice of the target question from the N first document slices based on the first matching degree between each first document slice and the target question; The apparatus further includes a construction unit, which is configured to acquire a target type document set, slice each target type document according to the characteristics of the target type documents to obtain at least one second document slice for each target type document, the target type document set including multiple target type documents; for each second document slice, combine the second document slice with the title to which the second document slice belongs to form a first document slice, and determine the feature value corresponding to the first document slice, and then construct the document slice library according to the feature value corresponding to each first document slice.
9. An electronic device, characterized in that, include: Processor and memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the question-and-answer method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
A method and apparatus for answer extraction
CN109189894A