Data and power data query method, device, storage medium and program product
By obtaining query requirements description information in the intelligent information service, determining the target document fragment and enhancing prompt information based on the metadata type, combined with the language model processing, the problem of low data query accuracy in the intelligent information service is solved, and efficient and accurate Data query results.
Patent Information
- Application Number
- CN202411013251.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2025-01-14
- Estimated Expiration
- 2044-07-26
AI Technical Summary
The prior art is difficult to improve query accuracy in the process of data query in intelligent information services, especially in the target field, the problem of low full retrieval efficiency of documents and incomplete document fragment semantics.
By obtaining query requirements description information, the target document fragment is determined, and the prompt information is enhanced based on the metadata type, the target document fragment is processed to improve the accuracy of the query results. Specific steps include layout element recognition of document fragments, structure tree generation, sliding window segmentation, vectorization processing and metadata enhancement prompt information application, combined with language model processing to improve the accuracy of query results.
It significantly improves the accuracy and efficiency of data queries, ensures the semantic integrity and context coherence of query results, and enhances the guidance of target document fragment processing.
Smart Images

Figure CN118550996B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data and power data query method, device, storage medium and program product. Background Art
[0002] With the development of artificial intelligence, intelligent information services continue to upgrade, and data query is required in various intelligent information service fields, such as intelligent question and answer, personalized push and information retrieval. In this intelligent information service, the intelligent information service system can perform data query based on the query demand information provided by the user, and output the corresponding query results to the user. The accuracy of data query has an important impact on the output results. Therefore, how to improve the accuracy of data query is a technical problem that technicians in this field continue to pay attention to and study. Summary of the invention
[0003] Various aspects of the present application provide a data and power data query method, device, storage medium and program product to improve data query accuracy.
[0004] The present application provides a data query method, including:
[0005] Get query requirement description information;
[0006] Determine a target document segment from documents in a target domain according to the query requirement description information;
[0007] Determine target prompt information according to the target metadata type associated with the query requirement description information; the target prompt information is used to describe a processing method for the target metadata type; the target metadata type is at least one metadata of the document;
[0008] According to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain a query result.
[0009] The present application also provides a method for querying power data, including:
[0010] Obtain query requirement description information for power data;
[0011] Determining a target document segment from the power document according to the query requirement description information;
[0012] Determine target prompt information according to the target metadata type associated with the query requirement description information; the target prompt information is used to describe a processing method for the target metadata type; the target metadata type is at least one metadata of the power document;
[0013] According to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain a query result.
[0014] The embodiment of the present application further provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store a computer program;
[0015] The processor is coupled to the memory, and is configured to execute the computer program to execute the steps in the above-mentioned data query method and / or power data query method.
[0016] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data query method and / or power data query method.
[0017] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data query method and / or power data query method.
[0018] In the embodiment of the present application, according to the query requirement description information, the target document fragment is determined from the documents in the target field, and the prompt information (Prompt) is enhanced based on the metadata associated with the query requirement description information. After that, the target document fragment is processed based on the processing method of the target metadata type described by the target prompt information enhanced by the metadata to obtain the query result. Among them, the processing method of the target metadata type described by the target prompt information enhanced by the metadata can better guide the processing of the target document fragment, which helps to improve the accuracy of the determined query result. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0020] Figure 1 A flowchart of a data query method provided in an embodiment of the present application;
[0021] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;
[0022] Figure 3a and Figure 3b A schematic diagram of a vector query process provided in an embodiment of the present application;
[0023] Figure 4aA schematic diagram of a flow chart of a power data query method provided in an embodiment of the present application;
[0024] Figure 4b A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0026] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0027] In some embodiments of the present application, according to the query requirement description information, the target document fragment is determined from the documents in the target field, and the prompt information (Prompt) is enhanced based on the metadata associated with the query requirement description information. After that, the target document fragment is processed based on the processing method of the target metadata type described by the target prompt information enhanced by the metadata to obtain the query result. Among them, the processing method of the target metadata type described by the target prompt information enhanced by the metadata can better guide the processing of the target document fragment, which helps to improve the accuracy of the determined query result.
[0028] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0029] It should be noted that the same reference numerals denote the same objects in the following drawings and embodiments, and therefore, once an object is defined in one drawing or embodiment, it does not need to be further discussed in the subsequent drawings and embodiments.
[0030] Figure 1 The following is a flow chart of a data query method provided in an embodiment of the present application. Figure 1 As shown, the data query method mainly includes:
[0031] 101. Obtain query requirement description information.
[0032] 102. According to the query requirement description information, determine the target document fragment from the documents in the target domain.
[0033] 103. Determine target prompt information according to the target metadata type associated with the query requirement description information; the target prompt information is used to describe a processing method for the target metadata type, and the target metadata type is at least one metadata type of documents in the target field.
[0034] 104. According to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain the query result.
[0035] In this embodiment, the target field can be a vertical field, which refers to a field that provides specific services to a limited group, such as electricity, industry, e-commerce, entertainment, medical care, environmental protection, education or sports. The target field only focuses on a certain part of an industry. Compared with the general field, the target field is more professional and has a deeper knowledge depth.
[0036] The document of the target field records the data of the target field in detail and can reflect the knowledge of the target field. For example, the document of the power field can reflect the knowledge of various parts involved in the power field, such as equipment conditions, power generation conditions and operation manuals, but is not limited to this. In the embodiment of the present application, the specific format of the document is not limited. The document can be text (such as .txt format), table, slide, picture or portable document format (PDF), etc.
[0037] The documents in the target domain are collected in advance, and the documents are highly structured, and the content is large and rich. When performing data query in the target domain, it is necessary to search in the documents in the target domain. When performing data query for the target domain, the user can provide query requirement description information according to the query requirement. Among them, the query requirement description information refers to the data describing the query requirement, which can be a text describing the query requirement in natural language, etc. The query requirement description information may include: query conditions and query intent or purpose, etc. Accordingly, when querying online data, in step 101, the query requirement description information can be obtained. Further, in step 102, the target document fragment can be determined from the documents in the target domain according to the query requirement description information.
[0038] In some embodiments, query conditions may be obtained from the query requirement description information, and the query conditions may be used to perform data search in documents in the target field to obtain target document fragments.
[0039] However, when performing a full search in the entire document of the target field, the amount of data retrieved is large, which greatly affects the data query efficiency. In an embodiment of the present application, in order to improve the data query efficiency, the documents in the target field can be segmented into fragments to obtain multiple document fragments. Multiple refers to 2 or more, which is specifically determined by the number of actual document segments. The number of general document fragments is tens of thousands, even hundreds of millions, or even more. When searching for data, searching in document fragments can reduce the amount of retrieved data and improve data query efficiency.
[0040] In the embodiments of the present application, the specific implementation method for segmenting the documents in the target field is not limited. Considering that the title of the document is generally short, therefore, only the body of the document can be segmented, which requires first identifying the layout elements of the document. Among them, the layout elements of the document are the overall expression of the document layout, which can reflect the structure of the document, and may include: text, title, picture, column, notes and chart descriptions and other elements, reflecting the overall structure of the document. Among them, the title of the document may include multi-level titles, such as main title, first-level title, second-level title, third-level title and even more levels of titles. Multi-level refers to two levels or more.
[0041] Based on this, Figure 2 As shown in "Layout Element Recognition", in an embodiment of the present application, layout element recognition can be performed on documents in the target field to determine the layout element recognition results of the document. Among them, the layout element recognition results of the document may include: the position information of the layout elements of the document and the type of the corresponding layout elements. The position information of the layout elements of the document is used to indicate the position of a certain part in the document, and the type of layout elements is used to identify which layout element a certain part of the document belongs to. For example, the layout element recognition results of the document are: the first-level title, the first line of the first page of the document, etc.; the main text, the third line of the second page to the fourth line of the third page of the document, etc.
[0042] In some embodiments, layout element recognition can be performed on documents in the target domain using a pre-trained layout element recognition model, wherein the layout element recognition model can be a neural network model or an optical character recognition (OCR) model.
[0043] When training the layout element recognition model, document samples in the target domain can be obtained, and layout elements can be annotated on the document samples, that is, publication elements such as the main title (first-level title), chapter title (second-level title), subsection title (third-level title) and text can be annotated on the document samples. Further, the document samples annotated with publication elements are used as training samples to train the initial model of the layout element recognition model, thereby obtaining a layout element recognition model.
[0044] Based on the pre-completed layout element recognition model, the documents in the target field can be input into the layout element recognition model; in the layout element recognition model, the layout elements of the documents in the target field are recognized to obtain the layout element recognition results of the documents in the target field.
[0045] Since the layout element recognition model is trained with document samples that are pre-labeled with version elements, the layout element recognition model learns the structure of the document during the training process. Therefore, when using the trained layout element recognition model to perform layout element recognition on documents in the target field, not only can character recognition be achieved, but also the understanding of the document structure is emphasized. The model can recognize layout elements of different levels.
[0046] Further, the body of the document in the target field is determined based on the layout element recognition result of the document in the target field. Specifically, the position information of the body of the document in the target field can be obtained from the layout element recognition result of the document in the target field; and the body of the document in the target field is determined based on the position information of the body of the document in the target field.
[0047] In some embodiments, the body of the document may be segmented into multiple document segments according to a set text length, each document segment having a set segment length. The document segments segmented in this document segmentation method may be semantically incomplete and fail to take into account the coherence of the context.
[0048] In order to ensure the semantic integrity of the document fragments and take into account the coherence of the context, a sliding window method can be used to segment the body of the document into multiple document fragments whose text length does not exceed the set text length. In this embodiment, the size of the sliding window used in the sliding window method is less than or equal to the set text length, and the step size of the sliding window is less than the set text length.
[0049] Specifically, starting from the first character of the body of the document, the text data can be selected using a sliding window, sliding one step each time, and obtaining the text data currently selected by the sliding window; then, it can be determined whether the semantics of the text data currently selected by the sliding window is complete. Specifically, a pre-trained neural network model for semantic integrity analysis can be used to determine the semantic integrity of the text data currently selected by the sliding window. If the semantics of the text data currently selected by the sliding window is incomplete, the sliding window is moved back by one word until the text semantics of the text data currently selected by the sliding window is complete; and the text data currently selected by the sliding window when the text semantics are complete is divided into a document segment. The body of the document can be divided into multiple document segments using the same method.
[0050] It should be noted that in each embodiment of the present application, the text length can be measured by the number of basic data units supported by the neural network model for processing, such as the text length can be measured by the number of words, etc. In some embodiments, for the neural network model, token is the basic data unit supported by the semantic vector model for processing, and is the smallest data measurement unit supported by the neural network model. Accordingly, the size and step size of the aforementioned sliding window can also be represented by the number of basic data units supported by the semantic vector model for processing. The set text length is 400 tokens. The size of the sliding window can be less than 400 tokens, and less than or equal to the set text length; the step size of the sliding window is less than the number of tokens in the sliding window.
[0051] Since the step size of the sliding window is smaller than the size of the sliding window, the beginning and end of the text data selected twice before and after the sliding window overlap, that is, there is a certain data overlap in the segmented document fragments. Therefore, the document fragments segmented using the sliding window method take into account the context of the document fragment, and also take into account the coherence of the document context.
[0052] When a number of document fragments are actually cut out from the body of a document in the target field for data query, the data query results are sometimes inaccurate. The inventor of the present application has studied and analyzed and found that the document fragments directly cut out from the body of the document in the target field exist independently. Sometimes an independent document fragment cannot reflect information related to the query demand, but the document fragment is actually related to the query demand. For example, in the power field, the information recorded in a certain document fragment is "The working environment temperature of equipment A is 40°C on March 20, 2018", and the query demand is: "What is the cause of the failure of power station B on March 20, 2018?" Judging from the content of the document fragment alone, it has nothing to do with power station B and the cause of the failure, and the main title of the document to which the actual document fragment belongs is: 2018 B Power Station Failure Analysis Report, and the title of the section to which the body of the document fragment belongs is "The working environment temperature of low-temperature equipment is too high". Judging from the main title of the document and the title of the section to which the body of the document fragment belongs, the document fragment is closely related to the query demand.
[0053] Based on the above analysis, in some embodiments of the present application, relevant structural path information can be added to the document fragments obtained by segmenting the body of the document using the sliding window method in the above embodiments. The structural path information can be understood as the index path or index directory of the document fragment in the document, which may include: the title and parent title of the body to which the document fragment belongs. For the convenience of description and distinction, in the embodiments where it is necessary to add relevant structural path information to the document fragments obtained by directly segmenting the body of the document in the target domain, the document fragments obtained by directly segmenting the body of the document in the target domain are defined as initial document fragments.
[0054] Based on the above analysis, we also need to obtain the structural path information of the initial document fragment. Specifically, Figure 2 As shown in "Structure tree generation", before segmenting the target domain document, the document structure tree can be generated according to the layout element recognition results of the target domain document. The document structure tree describes the hierarchical logic and content organization structure of the document in a tree form.
[0055] Specifically, the position information of the layout elements of the document in the document can be determined based on the results of the layout element recognition of the document; the dependency relationship between the layout elements of the document can be determined based on the position information of the layout elements of the document in the document. The dependency relationship between the layout elements of a document refers to which layout element belongs to which upper-level layout element. For example, which secondary title belongs to which primary title, etc. For example, a document contains 1 main title, 3 primary titles A, B, and C, and 5 secondary titles D, E, F, G, and H. Determining the dependency relationship between the layout elements is to determine which primary titles the 5 secondary titles D, E, F, G, and H belong to.
[0056] Specifically, based on the position information of the document's layout elements in the document, two adjacent layout elements of the same level can be determined, and other layout elements between the two adjacent layout elements of the same level (defined as target layout elements) can be determined; then, it can be determined that the target layout element belongs to the layout element that is located earlier in the document among the two adjacent layout elements of the same level. Using the same method, the dependency relationship between the layout elements of the document can be obtained.
[0057] Furthermore, a structure tree of the document can be generated according to the hierarchical relationship between the layout elements of the document and the dependency relationship between the version elements. Specifically, according to the hierarchical relationship between the layout elements of the document and the dependency relationship between the layout elements, the structure tree of the document can be obtained by traversing from the highest level title to the body paragraphs step by step, and taking each traversed layout element as a node of the structure tree.
[0058] Each node in the structure tree represents a layout element of the document, such as chapters, subsections, and text, and the connections between nodes reflect the inclusion relationship between them. The structure tree not only helps to understand the macro framework of the document, but also provides a clear basis for subsequent document segmentation.
[0059] Based on the structure tree of the document in the target domain, the structure path information of the text to which the initial document fragment belongs can be obtained from the structure tree of the document as the structure path information corresponding to the initial document fragment.
[0060] In combination with the structural path information of the initial document fragments, when dividing the documents in the target field, the target text length of each of the multiple initial document fragments to be segmented can be determined based on the text length of the structural path information of the body of the document and the set text length. Among them, the target text length can be the difference between the set text length and the text length of the structural path information of the body of the document. For texts with different text lengths of structural path information, the target text lengths of the segmented initial document fragments are different. For any body of a document, the target text length of the initial document fragment to be segmented from the body can be the difference between the set text length and the text length of the structural path information of the body. Among them, the measurement method of text length can refer to the relevant content of the aforementioned embodiment, which will not be repeated here.
[0061] Further, the body of the document is segmented into multiple initial document segments whose text length does not exceed the target text length using the sliding window method. For the specific implementation of segmenting the body of the document using the sliding window method, please refer to the relevant content of the aforementioned embodiment, which will not be repeated here. Further, the corresponding structural path information can be added to the initial document segments to obtain multiple document segments in the target field. The specific implementation of document segmentation described in the aforementioned embodiment corresponds to Figure 2 "Document segmentation". In this process, the corresponding structural path information is added to the initial document fragments, so that when the document fragments are processed independently, their positions and contexts in the original document can be traced, which can provide important clues for subsequent semantic understanding and retrieval, and help improve the accuracy of subsequent data queries.
[0062] After obtaining several document fragments in the target domain, these document fragments can be stored. When performing data query for the target domain, the user can provide query requirement description information according to the query requirement. The query requirement description information refers to data describing the query requirement, which can be text describing the query requirement in natural language. The query requirement description information can include: query conditions and query intent or purpose. Accordingly, when querying online data, in step 101, the query requirement description information can be obtained.
[0063] Based on several document fragments of the target domain obtained in advance, data query can be performed in these document fragments. Accordingly, the above step 102 can be implemented as follows: calculating the matching degree between the query requirement description information and the multiple document fragments of the target domain. The multiple document fragments of the target domain are obtained by segmenting the documents of the target domain, wherein the higher the matching degree between the document fragment and the query requirement description information, the more the document fragment meets the query requirement. Therefore, the target document fragment can also be determined from the multiple document fragments according to the matching degree between the query requirement description information and the multiple document fragments.
[0064] In the embodiments of the present application, the specific implementation method of calculating the matching degree between the query requirement description information and multiple document fragments is not limited. The following is an exemplary description of the specific implementation method of calculating the matching degree between the query requirement description information and multiple document fragments in the target field, and determining the target document fragment from multiple document fragments based on the matching degree between the query requirement description information and multiple document fragments, in combination with several optional implementation methods.
[0065] Embodiment 1: In some embodiments, as Figure 2 As shown in "Document Fragment Vectorization", several document fragments in the target field can be vectorized in advance to obtain the vectors of the document fragments in the target field. Specifically, the pre-trained semantic vector model can be used to vectorize the document fragments in the target field to obtain the vectors of each document fragment. Among them, the semantic vector model is obtained by model training using the corpus data of the target field as a training sample. In the embodiment of the present application, the specific implementation form of the semantic vector model is not limited. Optionally, the semantic vector model can be a general embedding (GE) model, an embedded language model (Embedding from language model, ELMo model), a generative pre-train model (GPT) model, or a bidirectional encoder representation (BERT) model based on a transformer. Among them, the semantic vector model can capture the deep semantic features of the text, map the text content to a continuous vector control, and similar semantic content is close in the vector space.
[0066] It should be noted that in implementation mode 1, the document fragments are vectorized by the semantic vector model. Therefore, in the aforementioned embodiment of segmenting the document, the set text length can be determined by the maximum text length supported by the semantic vector model for input. Specifically, the text length can be measured by the number of basic data units supported by the semantic vector model, such as the text length can be measured by the number of words, etc. For the semantic vector model, token is the basic data unit supported by the semantic vector model for processing, and is the smallest data measurement unit supported by the semantic vector model. Accordingly, the size and step size of the aforementioned sliding window can also be represented by the number of basic data units supported by the semantic vector model for processing. For example, the maximum text length of the input limit of the semantic vector model is 512 tokens, and the set text length can be less than or equal to 512 tokens. The size of the sliding window can be less than 512 tokens, and less than or equal to the set text length; the step size of the sliding window is less than the number of tokens in the sliding window.
[0067] Furthermore, document fragments of the target domain and vectors of the document fragments may be stored. Based on the vectors of the document fragments, when calculating the matching degree between the query requirement description information and the document fragments, the similarity between the query vector corresponding to the query requirement description information and the vector of the document fragment may be calculated. Accordingly, one implementation of step 102 is: vectorizing the query requirement description information to obtain a query vector corresponding to the query requirement description information; and calculating the similarity between the query vector and the vectors of multiple document fragments stored in advance as the matching degree between the query requirement description information and the multiple document fragments. Among them, the vector of the query requirement description information is referred to as the query vector corresponding to the query requirement description information. Optionally, the aforementioned semantic vector model may be used to vectorize the query requirement description information to obtain a query vector corresponding to the query requirement description information. Among them, the semantic vector model for vectorizing the query requirement description information is the same model as the aforementioned semantic vector model for vectorizing the document fragments of the target domain.
[0068] In some embodiments, the distance between the query vector and the vector of the document fragment may be calculated to measure the similarity between the query vector and the document fragment. The greater the distance between the query vector and the vector of the document fragment, the lower the similarity between the query vector and the vector of the document fragment. Optionally, the distance between the vectors may be cosine distance, Euclidean distance, Mahalanobis distance, Mahatton distance, or Hamming distance, but is not limited thereto.
[0069] Accordingly, step 103 can be implemented as follows: according to the similarity between the query vector and the vectors of multiple pre-stored document fragments, the first document fragment whose similarity between the corresponding vector and the query vector is in the top N positions is determined from the multiple document fragments as a candidate document fragment, that is, the similarity between the vector of the first document fragment (i.e., the candidate document fragment) and the query vector is in the top N positions. N ≥ 2 and is an integer. For example, N = 10, 15, 20, 25 or 30, etc. Alternatively, the first document fragment whose similarity between the corresponding vector and the query vector is greater than or equal to the set similarity threshold can be determined from the multiple document fragments as a candidate document fragment. Further, the target document fragment can be determined based on the first document fragment (i.e., the candidate document fragment).
[0070] Considering that if the query vector is matched one by one in the vectors of the pre-stored document fragments, the complexity is high and the query efficiency is low. In order to improve the query efficiency, in some embodiments, the approximate nearest neighbor algorithm can be used to perform spatial division on the vectors of the document fragments to obtain a multi-layer vector control corresponding to the document fragments. Among them, the approximate nearest neighbor (ANN) algorithm can be the Annoy algorithm, the locality sensitive hashing algorithm, the vector quantization algorithm or the K nearest neighbor (KNN) algorithm, etc., but is not limited thereto.
[0071] Accordingly, based on the multi-layer vector space corresponding to the above document fragment, when calculating the similarity between the query vector and the vector of the document fragment, the target subspace to which the query vector belongs can be determined from the multi-layer vector space corresponding to the document fragment according to the specified number N of candidate document fragments; further, a candidate vector is selected from the vectors included in the target subspace, and the document fragment corresponding to the candidate vector is determined to be the first document fragment (i.e., the candidate document fragment). This query method can quickly find the target subspace to which the query vector belongs, which helps to improve query efficiency.
[0072] Furthermore, the similarity between each vector contained in the target subspace and the query vector can be calculated, and according to the similarity between each vector contained in the target subspace and the query vector, a candidate vector is selected from the vectors contained in the target subspace, and the document segment corresponding to the candidate vector is determined to be the first document segment (i.e., the candidate document segment). In this way, the query vector only needs to perform similarity calculation with the vector in the target subspace, which can reduce the calculation complexity and help to further improve the query efficiency.
[0073] Specifically, according to the similarity between each vector included in the target subspace and the query vector, the candidate vectors with the top N similarities can be selected from the vectors included in the target subspace; or, from the vectors included in the target subspace, the candidate vectors with the similarity between the query vector and the target subspace greater than or equal to a set similarity threshold can be selected. Further, the document segment corresponding to the candidate vector can be determined to be the first document segment (i.e., the candidate document segment).
[0074] The following uses the Annoy algorithm as an example to exemplify a specific implementation method of spatially partitioning the vectors of document fragments and searching the query vector in the vectors of document fragments.
[0075] The Annoy algorithm can build the vectors of document fragments into a binary tree, that is, the multi-layer vector space has a binary tree structure, and then when the query vector is searched in the vectors of document fragments, the time complexity of the search is reduced to O(logR). Here, R is the number of vectors of document fragments. Figure 3a and Figure 3b, using the Annoy algorithm as an example, the process of spatially partitioning the vectors of document fragments is explained. Figure 3a and Figure 3b In order to facilitate the diagram, dots are used to represent the vectors of document fragments.
[0076] Step 1: Randomly select two vectors from the vectors of the document fragment, and use these two vectors as the initial center nodes, perform the K-means clustering process with a cluster number of 2, and finally generate two converged cluster center vectors, such as Figure 3a As shown in the five-pointed star.
[0077] Step 2: The two cluster center vectors form a hyperplane ( Figure 3a As shown by the dashed line in the middle), establish an equidistant perpendicular hyperplane to the hyperplane, as Figure 3a As shown by the solid line in the middle, the isometric vertical hyperplane divides the entire space composed of the vectors of the document fragments into two parts, A and B, that is, into two subspaces A and B. That is, the isometric vertical hyperplane is the hyperplane 1 corresponding to the subspaces A and B.
[0078] Step 3: Use the method in step 2 to continue recursively dividing the divided subspace until the number of vectors contained in each subspace is less than or equal to N.
[0079] At this point, the vectors of the document fragments are divided into multiple layers of vector space, and the multiple layers of vector space form a binary tree. The root node of the binary tree represents the full space of the vectors of the document fragments, and the left and right subtrees corresponding to the same parent node in each layer of the binary tree represent the two subspaces into which the vector space corresponding to the parent node is divided. For example, Figure 3b As shown, hyperplane 2 further divides subspace A into subspaces A1 and A2, and hyperplane 3 further divides subspace B into subspaces B1 and B2. Then, the left and right subtrees corresponding to parent node A represent subspaces A1 and A2 respectively; the left and right subtrees corresponding to parent node B represent subspaces B1 and B2 respectively.
[0080] In the embodiment of the present application, the vector space corresponding to the parent node is divided into two hyperplanes of subspaces, which are defined as the hyperplanes corresponding to the two subspaces. That is, the hyperplane corresponding to subspaces A1 and A2 is hyperplane 2, and the hyperplane corresponding to subspaces B1 and B2 is hyperplane 3.
[0081] Based on the above binary tree, when determining the target subspace to which the query vector belongs from the multi-layer vector space, the traversal direction of the binary tree can be determined according to the geometric relationship between the query vector corresponding to the query requirement description information and the hyperplanes corresponding to the left and right subtrees of each layer on the binary tree, and the binary tree is traversed along the traversal direction until the target subtree whose number of vectors contained in the represented subspace is less than or equal to N is found; and the subspace represented by the parent node of the target subtree is used as the target subspace to which the query vector belongs. Among them, the target subtree whose number of vectors contained in the represented subspace is less than or equal to N refers to: the subtree whose number of vectors contained in the represented subspace is less than or equal to N that appears for the first time during the traversal process.
[0082] For example, Figure 3a and Figure 3b As shown, subspaces A and B represent the left and right subtrees of the root node respectively; subspaces A1 and A2 represent the left and right subtrees of the parent node A respectively; subspaces B1 and B2 represent the left and right subtrees of the parent node B respectively; when determining the traversal direction of the query vector to the binary tree, the geometric relationship between the query vector and the hyperplane 1 corresponding to the subspaces A and B can be determined first; further, based on the geometric relationship between the query vector and the hyperplane 1 corresponding to the subspaces A and B, determine whether the query vector belongs to subspace A or subspace B. Furthermore, assuming that the query vector belongs to subspace A, the geometric relationship between the query vector and the hyperplane 2 corresponding to the subspaces A1 and A2 is further determined, and based on the geometric relationship, determine whether the query vector belongs to subspace A1 or the answer subspace A2. Furthermore, assuming that the query vector belongs to subspace A2, the traversal direction of the binary tree is: the entire space formed by the vectors of the document fragments -> answer subspace A -> answer subspace A2, and traverse in sequence in the above manner until a subspace containing vectors less than or equal to N is found, and the upper subspace of the subspace containing answer vectors less than or equal to N is used as the target subspace to which the query vector belongs. Further, a candidate vector is determined from each vector contained in the target subspace, and the document fragment corresponding to the candidate vector is determined to be the first document fragment (i.e., the candidate document fragment).
[0083] Furthermore, the target document fragment may be determined based on the first document fragment (i.e., the candidate document fragment). In some embodiments, the first document fragment (i.e., the candidate document fragment) may be used as the target document fragment. In other embodiments, in order to improve the accuracy of data query, the first document fragment (i.e., the candidate document fragment) may be further screened.
[0084] Specifically, feature extraction can be performed on the first document fragments (i.e., candidate document fragments) of the target domain to obtain multi-dimensional features corresponding to the first document fragments (i.e., candidate document fragments). The multi-dimensional features of the first document fragment refer to information reflecting the text characteristics of the first document fragment from multiple dimensions, and may include: keywords, knowledge words, language model features, semantic vectors of keywords of the first document fragment (i.e., candidate document fragment), and the part of speech of keywords of the candidate document fragment, but are not limited thereto.
[0085] In some embodiments, keywords of the target field may be collected in advance. Based on the keywords of the target field collected in advance, the document fragments may be segmented to obtain the words contained in the first document fragment (i.e., the candidate document fragment), and the words contained in the first document fragment (i.e., the candidate document fragment) are matched with the keywords of the target field collected in advance. If the word exists in the keywords of the target field, the word is extracted from the first document fragment (i.e., the candidate document fragment) as one of the keywords. In the same way, the keywords of the target field may be extracted from each first document fragment (i.e., the candidate document fragment).
[0086] The knowledge words of the first document fragment (i.e., the candidate document fragment) refer to the jargon of the target field contained in the document fragment, and have specific meanings in the target field. For example, in the field of power, the knowledge words include ranking, climbing, dancing, and bottle mouth. Among them, ranking, the full name of pole tower ranking, is the work of line engineers to arrange the tower positions of line pole towers according to the meteorological conditions, terrain conditions, obstacle distribution and line pole tower design conditions of the line using ranking software. Line engineers need to place each tower in a safe and reliable position, and at the same time ensure that the actual use conditions of each tower will not exceed the design conditions. Climbing, in power engineering, generally refers to field surveys, especially often refers to the extremely important "tower positioning" work in line design work. Line engineers need to go to the tower position of each pole tower for field surveys to determine the position of the pole tower. Dancing, in power engineering, refers to a low-frequency, large-amplitude self-excited vibration phenomenon caused by wind excitation after the eccentric ice coating of overhead transmission line conductors. Because of its shape of flying up and down, it looks like a dragon dance, it is called dancing. The bottle mouth, in power engineering, generally refers to the part below the phase tower window in a cathead or wine glass-shaped transmission tower. In places with large terrain height differences, the transmission towers located at high places have steeper downward inclinations, and the live conductors are prone to get too close to the bottle mouth, resulting in the line insulation gap not meeting the design specification requirements.
[0087] The knowledge words of the target domain may be collected in advance. Based on the knowledge words of the target domain collected in advance, the first document fragment (i.e., the candidate document fragment) may be segmented to obtain the words contained in the first document fragment (i.e., the candidate document fragment), and the words contained in the first document fragment (i.e., the candidate document fragment) may be matched with the knowledge words of the target domain collected in advance. If the word exists in the knowledge words of the target domain, the word is extracted from the first document fragment (i.e., the candidate document fragment) as one of the knowledge words. In the same way, the knowledge words of the target domain may be extracted from the first document fragment (i.e., the candidate document fragment).
[0088] The language model feature may be an n-gram feature, which represents n consecutive words or characters in a text. A feature vector may be constructed by counting the frequencies of occurrence of different n-grams in the first document segment (ie, the candidate document segment).
[0089] For the semantic vector of the keyword, the keyword of the first document segment (i.e., the candidate document segment) extracted above can be vectorized to obtain the semantic vector of the keyword. For example, the semantic vector model pre-trained above can be used to vectorize the keyword of the first document segment (i.e., the candidate document segment) to obtain the semantic vector of the keyword.
[0090] As for the part of speech of the keyword, the keyword can be subjected to part of speech analysis to determine the part of speech of the keyword, that is, to analyze whether the keyword is a name, a verb, an adjective, or other parts of speech.
[0091] The multi-dimensional features shown in the above embodiments are only exemplary and not limiting. In some embodiments, the multi-dimensional features may be 10, 20, 30 or even more. Accordingly, feature extraction may also be performed on the query requirement description information to obtain the multi-dimensional features of the query requirement description information. The implementation method of feature extraction on the first document fragment (i.e., the candidate document fragment) may be adopted to extract features from the query requirement description information.
[0092] Furthermore, the correlation between the first document fragment (i.e., the candidate document fragment) and the query requirement description information can be calculated based on the multi-dimensional features of the first document fragment (i.e., the candidate document fragment) and the multi-dimensional features of the query requirement description information. Specifically, the similarity between the multi-dimensional features of the first document fragment (i.e., the candidate document fragment) and the multi-dimensional features of the query requirement description information can be calculated, wherein the greater the similarity between the multi-dimensional features of the first document fragment (i.e., the candidate document fragment) and the multi-dimensional features of the query requirement description information, the greater the correlation between the first document fragment (i.e., the candidate document fragment) and the query requirement description information.
[0093] Optionally, the distance between the multidimensional features of the first document fragment (i.e., the candidate document fragment) and the multidimensional features of the query requirement description information can be calculated to measure the similarity between the multidimensional features of the first document fragment (i.e., the candidate document fragment) and the multidimensional features of the query requirement description information. The greater the distance between the multidimensional features of the first document fragment (i.e., the candidate document fragment) and the multidimensional features of the query requirement description information, the lower the similarity between the multidimensional features of the first document fragment (i.e., the candidate document fragment) and the multidimensional features of the query requirement description information. Optionally, the distance between the multidimensional features of the first document fragment (i.e., the candidate document fragment) and the multidimensional features of the query requirement description information can be cosine distance, Euclidean distance, Mahalanobis distance, Mahalanobis distance, Hamming distance, etc., but is not limited thereto.
[0094] Furthermore, the target document fragment can be determined from the first document fragment (i.e., candidate document fragment) according to the correlation between the first document fragment (i.e., candidate document fragment) and the query requirement description information. Specifically, the first document fragment (i.e., candidate document fragment) whose correlation is in the top Q positions can be determined from the first document fragment (i.e., candidate document fragment) as the target document fragment according to the correlation between the first document fragment (i.e., candidate document fragment) and the query requirement description information. Where 0<Q<N, and is an integer. For example, N=30, Q can be 5, 10 or 15, etc.
[0095] In this embodiment, the similarity between the vector of the document fragment and the query vector corresponding to the query requirement description information is used to recall the document fragment vector, and the multi-dimensional characteristics of the document fragment and the multi-dimensional characteristics of the query requirement description information are used to measure the correlation between the document fragment and the query requirement description information for vector refinement. The vector recall and vector refinement are combined to screen the document fragments from different dimensions, enriching the reference information for data query, thereby helping to improve the accuracy of data query.
[0096] In implementation mode 1, the implementation mode of determining the first document fragment is as follows: Figure 2 The implementation method of determining the target document segment from the first document segment is as follows: Figure 2 "Vector Precision Sorting" in .
[0097] Implementation method 2: The text similarity between the query requirement description information and each document fragment in the target domain may be calculated as the matching degree between the query requirement description information and multiple document fragments in the template domain.
[0098] In some embodiments, a full-text search engine can be used to perform a full-text search on the query requirement description information in multiple document fragments in the target field. During the full-text search process, the full-text search engine is used to calculate the text similarity between the query requirement description information and the document fragments in the target field as the matching degree between the query requirement description information and the multiple document fragments in the target field.
[0099] Specifically, a full-text search engine can be used to perform word segmentation on multiple document fragments in the target domain to obtain the words (tokens) of each document fragment. For any document fragment, the non-repeated words in the document fragment can be composed into a word segmentation list and stored in the form of a file (Document). Accordingly, when calculating the text similarity between the query requirement description information and each document fragment in the target domain, the number of words (tokens) matching the query requirement description information and each document fragment can be calculated to measure the text similarity between the query requirement description information and each document fragment in the target domain. Among them, the more the number of words matching the query requirement description information and the document fragment, the higher the text similarity between the query requirement description information and the document fragment.
[0100] Accordingly, step 103 can be implemented as follows: according to the text similarity between the query vector and the vectors of the plurality of pre-stored document fragments, a second document fragment whose text similarity with the query vector is in the top M positions is determined from the plurality of document fragments as a candidate document fragment, i.e., the similarity between the vector of the second document fragment (i.e., the candidate document fragment) and the query vector is in the top M positions. M≥2 and is an integer. For example, M=10, 15, 20, 25 or 30, etc. M and N can be the same or different.
[0101] Furthermore, the target document fragment may be determined based on the second document fragment (i.e., the candidate document fragment). In some embodiments, the second document fragment (i.e., the candidate document fragment) may be used as the target document fragment. In other embodiments, in order to improve the accuracy of data query, the second document fragment (i.e., the candidate document fragment) may be further screened.
[0102] Specifically, feature extraction can be performed on the second document fragments (i.e., candidate document fragments) in the target domain to obtain multi-dimensional features corresponding to the second document fragments (i.e., candidate document fragments). For the specific implementation of the multi-dimensional features and feature extraction of the second document fragments, please refer to the aforementioned related content of feature extraction of the multi-dimensional features of the first document fragment, which will not be repeated here.
[0103] Accordingly, feature extraction may be performed on the query requirement description information to obtain multi-dimensional features of the query requirement description information. Feature extraction may be performed on the query requirement description information by adopting an implementation method of extracting features from the second document segment (ie, the candidate document segment).
[0104] Furthermore, the correlation between the second document fragment (ie, the candidate document fragment) and the query requirement description information may be calculated based on the multi-dimensional features of the second document fragment (ie, the candidate document fragment) and the multi-dimensional features of the query requirement description information.
[0105] Furthermore, the target document fragment can be determined from the second document fragment (i.e., the candidate document fragment) according to the correlation between the second document fragment (i.e., the candidate document fragment) and the query requirement description information. Specifically, the second document fragment with the correlation in the top Q positions can be determined from the second document fragment (i.e., the candidate document fragment) as the target document fragment according to the correlation between the second document fragment (i.e., the candidate document fragment) and the query requirement description information. Where 0<Q<N, and is an integer. For example, N=30, Q can be 5, 10 or 15, etc.
[0106] In this embodiment, the text similarity between the document fragment and the query requirement description information is used to perform vector recall on the document fragment, and the multi-dimensional features of the document fragment and the multi-dimensional features of the query requirement description information are used to measure the correlation between the document fragment and the query requirement description information for vector refinement. Vector recall and vector refinement are combined to screen the document fragments from different dimensions, enriching the reference information for data query, thereby helping to improve the accuracy of data query.
[0107] In implementation mode 2, the implementation mode of determining the second document fragment is as follows: Figure 2 The implementation method of determining the target document segment from the second document segment is as follows: Figure 2 "Vector Precision Sorting" in .
[0108] Implementation 3: The above implementation 1 and implementation 2 can be combined to calculate the matching degree between the query requirement description information and multiple document fragments in the target field. Specifically, the above implementation 1 is used to calculate the similarity between the query vector and the vectors of multiple document fragments stored in advance, and the above implementation 2 is used to calculate the text similarity between the query requirement description information and the multiple document fragments. Afterwards, the similarity between the query vector and the vectors of multiple document fragments stored in advance and the text similarity between the query requirement description information and the multiple document fragments can be used as the matching degree between the query requirement description information and the multiple document fragments.
[0109] Accordingly, step 103 can be implemented as follows: based on the similarity between the query vector and the vectors of multiple pre-stored document fragments, determine from the multiple document fragments the first document fragment whose similarity between the corresponding vector and the query vector is in the top N positions. Alternatively, determine from the multiple document fragments the first document fragment whose similarity between the corresponding vector and the query vector is greater than or equal to a set similarity threshold. And, based on the text similarity between the query vector and the vectors of multiple pre-stored document fragments, determine from the multiple document fragments the second document fragment whose text similarity with the query vector is in the top M positions. The first document fragment and the second document fragment are candidate document fragments. Thereafter, the target document fragment can be determined based on the first document fragment and the second document fragment.
[0110] In implementation mode 3, vector recall of document fragments is performed by utilizing the similarity between the vector of the document fragment and the query vector corresponding to the query requirement description information, and combining the text similarity between the document fragment and the query requirement description information. This can improve the comprehensiveness and diversity of the vector recall results and increase the possibility of finding the correct answer to the data query.
[0111] In some embodiments, the first document fragment and the second document fragment may be used as target document fragments. In other embodiments, in order to improve the accuracy of data query, the first document fragment and the second document fragment may be further screened.
[0112] Specifically, feature extraction can be performed on the first document fragment and the second document fragment of the target domain respectively to obtain the multi-dimensional features of the first document fragment and the multi-dimensional features corresponding to the second document fragment. For the specific implementation of the multi-dimensional features and feature extraction of the first document fragment and the second document fragment, please refer to the related content of feature extraction of the multi-dimensional features of the first document fragment, which will not be repeated here.
[0113] Accordingly, feature extraction may also be performed on the query requirement description information to obtain multi-dimensional features of the query requirement description information. The query requirement description information may be subjected to feature extraction in the same manner as the first document segment.
[0114] Furthermore, the correlation between the first document fragment and the query requirement description information and the correlation between the second document fragment and the query requirement description information can be calculated based on the multidimensional features of the first document fragment, the multidimensional features of the second document fragment and the multidimensional features of the query requirement description information.
[0115] Furthermore, the target document fragment can be determined from the first document fragment and the second document fragment according to the correlation between the first document fragment and the query requirement description information and the correlation between the second document fragment and the query requirement description information. Specifically, the document fragment with the top Q correlations can be determined from the first document fragment and the second document fragment as the target document fragment according to the correlation between the first document fragment and the query requirement description information and the correlation between the second document fragment and the query requirement description information. Wherein, 0<Q<N, and is an integer. For example, N=30, Q can be 5, 10 or 15, etc.
[0116] In this embodiment, the document fragments are recalled by combining the similarity between the vector of the document fragment and the query vector corresponding to the query requirement description information, and the text similarity between the document fragment and the query requirement description information, which can improve the comprehensiveness and diversity of the vector recall results and increase the possibility of finding the correct answer to the data query. On the other hand, the multi-dimensional features of the document fragment and the multi-dimensional features of the query requirement description information are used to measure the correlation between the document fragment and the query requirement description information for vector refinement. The vector recall and vector refinement are combined to screen the document fragments from different dimensions, enrich the reference information of the data query, and thus help improve the accuracy of the data query.
[0117] In implementation mode 3, the implementation mode of determining the first document segment and the second document is as follows: Figure 2 The implementation method of determining the target document segment from the first document segment and the second document segment is as follows: Figure 2 "Vector Precision Sorting" in .
[0118] Implementation 4: Extract features from multiple document fragments in the target domain to obtain multi-dimensional features corresponding to the multiple document fragments. For the specific implementation of the multi-dimensional features and feature extraction of document fragments, please refer to the above-mentioned related content of feature extraction of multi-dimensional features of document fragments, which will not be repeated here.
[0119] Accordingly, feature extraction may also be performed on the query requirement description information to obtain multi-dimensional features of the query requirement description information. The query requirement description information may be subjected to feature extraction in the same manner as the first document segment.
[0120] Furthermore, the correlation between the document fragment and the query requirement description information can be calculated based on the multi-dimensional features of the document fragment and the multi-dimensional features of the query requirement description information as the matching degree between the document fragment and the query requirement description information. Furthermore, the target document fragment can be determined from multiple document fragments in the target field based on the correlation between the document fragment and the query requirement description information.
[0121] In implementation mode 4, the multi-dimensional features of the document fragment and the multi-dimensional features of the query requirement description information are used to measure the correlation between the document fragment and the query requirement description information, wherein the more multi-dimensional features there are, the more accurate the determined target document fragment is.
[0122] The specific implementation methods of calculating the matching degree between a document fragment and query requirement description information, and the specific implementation methods of determining a target document fragment from multiple document fragments based on the matching degree between the document fragment and query requirement description information shown in the aforementioned embodiments are merely illustrative and do not constitute limitations.
[0123] After determining the target document fragment, the query result corresponding to the query requirement description information can be determined according to the target document fragment. In the embodiment of the present application, in order to improve the accuracy and quality of the query result, the processing process of the target document fragment can be enhanced with prompts. Specifically, in step 103, the target prompt information can be determined according to the target metadata type associated with the query requirement description information, that is, Figure 2 The target prompt information is used to describe the processing method of the target metadata type. The target metadata type is at least one metadata of the document in the target field.
[0124] The metadata of a document refers to the attribute information of the document, which is the data describing the document, and may include: the name of the document, the creator, the creation time, the generation time of the data recorded in the document, the department to which the document belongs, the data object corresponding to the document, the location, the structural information of the document, and the data form of the document. The data form of the document includes: numerical values, pictures, tables, and texts. Accordingly, the target metadata type may be one or more of these metadata of the document, and multiple refers to two or more.
[0125] In some embodiments, the query requirement description information may be subjected to query requirement type identification to determine the query requirement type reflected by the query requirement description information. The query requirement type may be classified according to the query purpose or intent. For example, the query requirement type may include: calculation type, table processing type, or specific document query type, etc. Optionally, a query requirement type identification model may be used to identify the query requirement type of the query requirement description information. The query requirement type identification model may be a classification model, such as a decision tree model, a support vector machine model, a random forest model, a neural network model, or a gradient boosting tree model, etc.
[0126] Furthermore, the target metadata type associated with the query requirement description information can be determined according to the query requirement type. Optionally, the correspondence between the query requirement type and the metadata type can be pre-configured. For example, if the query requirement type is a calculation type, the corresponding metadata type is that the data form is a numerical value; if the query requirement type is a table processing type, the corresponding metadata type is the guidance information for table processing. Among them, the guidance information for table processing is used to guide how to process the table. For example, the query requirement description information is: Please give the monthly power generation of new energy in region A in 2022 in the form of a table; then the query requirement type can be a table processing type, and the guidance information for table processing can be the format information of the table, such as how many rows and columns the table has, what is the name of each column, etc. If the query requirement type is a specific document query type, the corresponding metadata can be metadata that indicates the scope of the document query, such as the time information of the document, the name of the document, the department or region to which the document belongs, etc.
[0127] Based on the correspondence between the query requirement type and the metadata type, the query requirement type reflected by the query requirement description information can be matched in the correspondence; and the metadata type corresponding to the query requirement type reflected by the query requirement description information in the correspondence is determined to be the target metadata type.
[0128] In other embodiments, the target metadata type associated with the query requirement description information can be determined based on the metadata type of the target document fragment. Optionally, the metadata type involved in the target document fragment can be used as the target metadata type associated with the query requirement description information. For example, if the data format of the target document fragment is a table type, then the target metadata type associated with the query requirement description information is a table type. For another example, if the metadata of the target document fragment is time, such as the target document fragment is "Transformer failure on October 8, 2020", then the target metadata type may be time, etc.
[0129] In some other embodiments, the target metadata type associated with the query requirement description information may be determined according to the query requirement type reflected by the query requirement description information and the metadata type of the target document fragment. Specifically, the target metadata type determined according to the query requirement type reflected by the query requirement description information and the target metadata type determined according to the metadata type of the target document fragment may both be used as the target metadata type associated with the query requirement description information.
[0130] After determining the target metadata type associated with the query requirement description information, in step 103, target prompt information may be determined according to the target metadata type associated with the query requirement description information to implement document-based metadata enhanced prompt information (Prompt).
[0131] Specifically, a prompt information template may be configured in advance for each metadata type. When determining the target prompt information according to the target metadata type associated with the query requirement description information, the prompt information template corresponding to the target metadata type may be determined from the prompt templates corresponding to the pre-configured metadata type. The prompt information template corresponding to the target metadata type may be one or more. Multiple refers to two or more. For example, if the metadata quantity type is a data form of a numerical type, the corresponding prompt information template may be a numerical calculation method. There may be multiple numerical calculation methods. Multiple refers to two or more, which is specifically determined by actual application requirements. For another example, if the metadata type is a data form of a table type, the corresponding prompt information template may be processing guidance information for the table. For another example, if the metadata type is metadata for filtering specific documents, such as the time of the document (creation time or the generation time of the content described in the document, etc.), the name of the document or the region or department to which the document belongs, etc., the corresponding prompt information template may be information indicating the scope of the query document, such as querying the total power generation of a department on a certain day of a certain month of a certain year, etc.
[0132] Furthermore, a target prompt information template that meets the requirements of the query requirement description information is determined from the prompt information templates corresponding to the target metadata type, and the target prompt information template can be modified according to the query requirement description information to obtain the target prompt information.
[0133] Specifically, the prompt feature of the target metadata type can be determined according to the target navigation prompt template; the characteristic value of the prompt feature of the target metadata type is obtained from the query demand description information; the characteristic value of the prompt feature of the target metadata type is filled into the position corresponding to the prompt feature of the target metadata type in the target navigation prompt template to obtain the target prompt information. For example, the query demand description information is: What is the proportion of the power generation of new energy in region A to the total power generation in region A? Then the query demand type is a calculation type, and the corresponding target metadata type is a data form of a numerical value. The target prompt information template that meets the requirements of the query demand description information is: the power generation of a certain power generation method in a certain region / the total power generation in a certain region; the prompt feature of the target metadata type is: the power generation of a certain region and a certain power generation method; the characteristic values of the prompt feature of the target metadata type obtained from the query demand description information are: the characteristic value of a certain region is region A, and a certain power generation method is new energy, then the target prompt information is: the power generation of new energy in region A / the total power generation in region A.
[0134] Furthermore, in step 104, the target document fragment may be processed according to the processing method described in the target prompt information and the query requirement description information to obtain a query result corresponding to the query requirement description information.
[0135] In this embodiment, prompt information is enhanced based on metadata associated with the query requirement description information, and the target document fragment is processed based on the processing method of the target metadata type described by the target prompt information enhanced by the metadata, which can better guide the processing of the target document fragment and help improve the accuracy of the determined query results.
[0136] Specifically, Figure 2 As shown in "Language Model Processing", when the target document fragment is processed according to the processing method described in the target prompt information and the query requirement description information, the target prompt information is used as the prompt information (Prompt) of the language model, the query requirement description information is used as the question to be queried, and the target document fragment is used as the object to be processed. The target prompt information, the query requirement description information and the target document fragment are input into the language model. In the language model, the target document fragment is processed according to the processing method described in the target prompt information and the query requirement description information to obtain the query result corresponding to the query requirement description information.
[0137] In the embodiment of the present application, the language model is mainly used in the field of natural language processing (NLP). The goal of these models is to understand and generate human language. In order to achieve this goal, the model needs to be trained on a large amount of text data to learn various patterns and structures of the language. In this embodiment, the language model can output text information adapted to the prompt information under the trigger of the input prompt information.
[0138] In an embodiment of the present application, the language model may be a neural network model. In some embodiments, when the number of model parameters of the language model is large, for example, the number of parameters of the language model is millions, hundreds of millions, billions or even more, the language model may also be referred to as a large language model (LLM). In an embodiment of the present application, a large language model is defined as a neural network model whose number of model parameters conforms to a preset parameter number range. Among them, the number of model parameters corresponding to the preset parameter number range is very large, which may be millions, hundreds of millions, billions or even more, and the specific value may be determined by the standard in the field of artificial intelligence (AI).
[0139] Among them, the large language model can be trained through two stages, namely pre-training and fine-tuning. In the pre-training stage, the model is trained on large-scale general text data to learn the basic structure of the language and various common sense. Then, in the fine-tuning stage, the model is further trained on a smaller, more specific field data set. Fine-tuning allows the model to better understand and generate the language of this specific field (in the embodiment of the present application, the specific field is the target field), so as to better complete specific tasks.
[0140] In this embodiment, the large language model can use the historical query requirements in the target field as model input and the target prompt information determined by the metadata type associated with the historical query requirements as prompt information (Prompt) during the fine-tuning stage, and use the historical query results corresponding to the historical query requirements as the actual results corresponding to the model input for model training to obtain the language model provided in this embodiment, that is, the large language model. The loss function of the language model can be determined by the difference between the query result prediction information corresponding to the historical query requirements output by the language model during the model training stage and the actual results corresponding to the above model input (that is, the historical query results corresponding to the historical query requirements).
[0141] In the embodiments of the present application, the specific implementation architecture of the large language model is not limited. The large language model can adopt a language model architecture with permission to use and open source, or a self-developed language model architecture. Optionally, the large language model can be a generative pre-training model, such as a GPT (Generative Pre-Training) model), a general language model (General Language Model, GLM), or a variation of these models.
[0142] In this embodiment, the prompt information (Prompt) of the language model is enhanced through the metadata associated with the query requirement description information. In this way, the language model can be guided to focus more on the essence of the query problem, improve the ability to process the query requirement description information of a specific query requirement type, and help improve the accuracy and quality of the determined query results.
[0143] Combine the following Figure 2 , the data processing method provided in the embodiment of the present application is specifically described from the process of offline processing and online query.
[0144] like Figure 2 As shown, in the offline processing stage, the layout elements of the documents in the target field can be recognized to obtain the layout element recognition results of the documents (i.e. Figure 2 "Layout element recognition"). Afterwards, the document structure tree (i.e. Figure 2 "structure tree generation").
[0145] Further, if Figure 2 As shown in "Document Segmentation", the document can be segmented. Based on the structure tree of the document, the main text of the document is segmented into multiple initial document fragments whose text length does not exceed the target text length, wherein the target text length may be less than or equal to the difference between the set text length and the structural path information of the main text of the document. The set text length may be determined by the maximum number of words (such as the maximum number of tokens) supported by the subsequent semantic vector model for processing. Furthermore, the structural path information of the corresponding main text may be added to the multiple initial document fragments to obtain several document fragments in the target field. Among them, adding the structural path information to the initial document fragments enables the document fragments to be traced back to their position and context in the original document when they are processed independently, which can provide important clues for subsequent semantic understanding and retrieval, and help improve the accuracy of subsequent data queries.
[0146] Furthermore, the semantic vector model can be used to vectorize the document fragments obtained by document segmentation (i.e. Figure 2 "Document fragment vectorization"), obtain the vector of the document fragment, and store the vector of the document fragment. So far, the preliminary offline preparation work has been completed.
[0147] Based on the vectors of the document fragments in the stored target domain, vector recall can be performed during online data query. Specifically, Figure 2 As shown, the query requirement description information can be obtained, and the query requirement description information can be vectorized using the semantic vector model to obtain the query vector corresponding to the query requirement description information. The semantic vector model used to vectorize the query requirement description information is the same as the semantic vector model used to vectorize the document fragment. Further, the query vector corresponding to the query requirement description information can be used to perform vector recall on the vector of the document fragment (i.e. Figure 2 "Vector recall"). For specific vector recall methods, please refer to the above-mentioned implementation modes 1, 2 and 3.
[0148] Furthermore, the vector recall results can be sorted (i.e. Figure 2"Vector fine ranking"). When "Vector Recall" adopts the aforementioned implementation mode 1, fine ranking of the vector recall results means: calculating the correlation between the first document fragment and the query requirement description information based on the multi-dimensional features of the first document fragment and the multi-dimensional features of the query requirement description information; and determining the target document fragment from the first document fragment based on the correlation between the first document fragment and the query requirement description information. When "Vector Recall" adopts the aforementioned implementation mode 2, fine ranking of the vector recall results means calculating the correlation between the second document fragment and the query requirement description information based on the multi-dimensional features of the second document fragment and the multi-dimensional features of the query requirement description information; and determining the target document fragment from the second document fragment based on the correlation between the second document fragment and the query requirement description information. When the aforementioned implementation method 3 is adopted in the "vector recall", fine sorting of the vector recall results means: calculating the correlation between the first document fragment and the query requirement description information according to the multi-dimensional features of the first document fragment and the multi-dimensional features of the query requirement description information; calculating the correlation between the second document fragment and the query requirement description information according to the multi-dimensional features of the second document fragment and the multi-dimensional features of the query requirement description information; determining the target document fragment from the first document fragment and the second document fragment according to the correlation between the first document fragment and the query requirement description information and the correlation between the second document fragment and the query requirement description information.
[0149] After determining the target document fragment, metadata can be used to enhance the prompt information, that is, Figure 2 "Metadata enhancement prompt information". Specifically, the target metadata type associated with the demand description information can be queried. For specific methods, see the relevant content of the above embodiment. Further, the target prompt information can be determined according to the target metadata type associated with the query demand description information, and the metadata enhancement prompt information based on the document can be realized.
[0150] Further, if Figure 2 As shown in "Language Model Processing", the target prompt information is used as the prompt information (Prompt) of the language model, the query requirement description information is used as the query question, and the target document fragment is used as the object to be processed. The target prompt information, the query requirement description information and the target document fragment are input into the language model. In the language model, according to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain the query result corresponding to the query requirement description information.
[0151] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 and 102 can be device A; for another example, the execution subject of step 101 can be device A, and the execution subject of step 102 can be device B; and so on.
[0152] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.
[0153] The data query method provided in the embodiment of the present application can be applied to various fields, such as power, industry, e-commerce, entertainment, medical, environmental protection, education or sports, etc. The following is an exemplary description of the data query method applied to the power field.
[0154] Figure 4a The following is a flow chart of a method for querying power data provided by an embodiment of the present application. Figure 4a As shown, the power data query method mainly includes:
[0155] 401. Obtain query requirement description information for power data.
[0156] 402. Determine a target document segment from the power document according to the query requirement description information.
[0157] 403. Determine target prompt information according to the target metadata type associated with the query requirement description information; the target prompt information is used to describe a processing method for the target metadata type, and the target metadata type is at least one metadata type of documents in the target field.
[0158] 404. According to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain the query result.
[0159] The specific implementation of steps 401 to 404 can be found in the relevant contents of the above embodiment, which will not be described here. The power document corresponds to the document of the above target field, and the query requirement description information for power data corresponds to the query requirement description information of the above embodiment.
[0160] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data query method and / or power data query method.
[0161] The embodiment of the present application also provides a computer program product, including a computer program, which, when executed by one or more processors, causes the one or more processors to execute the steps in the above-mentioned data query method and / or power data query method. In the embodiment of the present application, the specific implementation form of the computer program product is not limited. In some embodiments, the computer program product can be implemented as an application (Application, APP), a computer-side client, a plug-in, a system software or a software as a service (Software as a Service, SaaS) form software, etc.
[0162] Figure 4b This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4b As shown, the electronic device includes: a memory 40a and a processor 40b. The memory 40a is used to store computer programs.
[0163] The processor 40b is coupled to the memory 40a and is used to execute the computer program to execute the steps in the data query method and / or power data query method provided in the above embodiments. The specific implementation of each step can be found in the relevant description of the above embodiments, which will not be repeated here.
[0164] In some optional embodiments, such as Figure 4b As shown, the computing device may also include optional components such as a communication component 40c, a power component 40d, a display component 40e, and an audio component 40f. Figure 4b The components are shown schematically only, and do not mean that the electronic device must include Figure 4b The components shown do not necessarily mean that the electronic device can only include Figure 4b Components shown.
[0165] in addition, Figure 4b The components in the dashed box are optional components, not mandatory components, and may depend on the product form of the electronic device. The electronic device of this embodiment may be implemented as a terminal device such as a desktop computer, a laptop computer, a mobile phone, or an IoT device; or it may be various server devices such as a traditional server, a cloud server, or a server cluster.
[0166] In an embodiment of the present application, the memory is used to store a computer program and can be configured to store various other data to support operations on the device where it is located. Among them, the processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random-Access Memory, SRAM), electrically erasable programmable read only memory (Electrically Erasable Programmable Read Only Memory, EEPROM), erasable programmable read only memory (Electrical Programmable Read Only Memory, EPROM), programmable read only memory (Programmable Read Only Memory, PROM), read only memory (Read Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0167] In the embodiment of the present application, the processor may be any hardware processing device that can execute the logic of the above method. Optionally, the processor may be a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it may also be a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD) or other programmable devices; or it may be an advanced reduced instruction set (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (System on Chip, SoC), etc., but not limited thereto.
[0168] In an embodiment of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it is located and other devices. The device in which the communication component is located can access a wireless network based on a communication standard, such as Wireless Fidelity (WiFi), 2G or 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology or other technologies.
[0169] In an embodiment of the present application, the display component may include a liquid crystal display (LCD) and a touch panel (TP). If the display component includes a touch panel, the display component may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0170] In an embodiment of the present application, a power supply component is configured to provide power to various components of the device in which it is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.
[0171] In an embodiment of the present application, the audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. For example, for a device with a language interaction function, voice interaction with a user may be achieved through an audio component.
[0172] It should be noted that the descriptions such as “first” and “second” in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit “first” and “second” to different types.
[0173] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program codes.
[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0177] In a typical configuration, a computing device includes one or more processors (CPU, etc.), input / output interfaces, network interfaces, and memory.
[0178] Memory may include non-permanent storage in a computer-readable medium, in the form of random-access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0179] Computer storage media is readable storage media, also known as readable media. Readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0180] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the above elements.
[0181] The above contents are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A data query method, characterized in that: include: Get query requirement description information; Determine a target document segment from documents in a target domain according to the query requirement description information; Determining target prompt information according to the target metadata type associated with the query requirement description information; The target prompt information is used to describe the processing method of the target metadata type; the target metadata type is at least one metadata of the document; According to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain a query result; Wherein, determining the target document fragment from the documents in the target field according to the query requirement description information includes: Calculating the matching degree between the query requirement description information and a plurality of document fragments in a target domain; the plurality of document fragments are obtained by segmenting documents in the target domain; Determine a candidate document segment from the multiple document segments according to the matching degree between the query requirement description information and the multiple document segments; the matching degree between the candidate document segment and the query requirement description information is in the top N positions; N ≥ 2 and is an integer; Performing feature extraction on the candidate document segments respectively to obtain multi-dimensional features of the candidate document segments; Performing feature extraction on the query demand description information to obtain multi-dimensional features of the query demand description information; Calculating the correlation between the candidate document fragment and the query requirement description information according to the multi-dimensional features of the candidate document fragment and the multi-dimensional features of the query requirement description information; The target document segment is determined from the candidate document segments according to the correlation between the candidate document segments and the query requirement description information.
2. The method according to claim 1, characterized in that The step of determining target prompt information according to the target metadata type associated with the query requirement description information includes: Determining a prompt information template corresponding to the target metadata type from the prompt information templates corresponding to the pre-configured metadata types; Determine, from the prompt information templates corresponding to the target metadata type, a target prompt information template that meets the requirement of the query requirement description information; The target prompt information template is modified according to the query requirement description information to obtain the target prompt information.
3. The method according to claim 1, characterized in that Also includes: Performing query requirement type identification on the query requirement description information to determine the query requirement type reflected by the query requirement description information; Determining, according to the query requirement type, a target metadata type associated with the query requirement description information; or, Determining a target metadata type associated with the query requirement description information according to the metadata type of the target document fragment; or, The query requirement type is identified on the query requirement description information to determine the query requirement type reflected by the query requirement description information; and the target metadata type associated with the query requirement description information is determined according to the query requirement type and the metadata type of the target document fragment.
4. The method according to any one of claims 1 to 3, characterized in that: The processing of the target document fragment according to the processing method described in the target prompt information and the query requirement description information to obtain the query result includes: Taking the target prompt information as the prompt information of the language model, taking the query requirement description information as the question to be queried, taking the target document fragment as the object to be processed, inputting the target prompt information, the query requirement description information and the target document fragment into the language model; In the language model, the target document fragment is processed according to the processing method described in the target prompt information and the query requirement description information to obtain the query result.
5. The method according to any one of claims 1 to 3, characterized in that: The calculating the matching degree between the query requirement description information and the document fragment of the target domain includes: Vectorizing the query requirement description information to obtain a query vector corresponding to the query requirement description information; calculating the similarity between the query vector and the pre-stored vectors of the multiple document fragments as the matching degree between the query requirement description information and the multiple document fragments; and / or, The text similarity between the query requirement description information and the multiple document fragments is calculated as the matching degree between the query requirement description information and the multiple document fragments.
6. The method according to any one of claims 1 to 3, characterized in that: Before calculating the matching degree between the query requirement description information and the multiple document fragments in the target domain, the method further includes: Performing layout element recognition on the document to determine a layout element recognition result of the document; Generate a structure tree of the document according to the recognition result of the layout elements of the document; Segmenting the documents in the target domain to obtain a plurality of initial document segments; Acquire, from the structure tree of the document, structure path information of the text to which each of the multiple initial document fragments belongs as structure path information corresponding to each of the multiple initial document fragments; The corresponding structural path information is added to the multiple initial document fragments to obtain the multiple document fragments.
7. The method according to claim 6, characterized in that The performing layout element recognition on the document to determine the layout element recognition result of the document includes: Inputting the document into a pre-trained layout element recognition model; wherein the layout element recognition model is obtained by training a model using document samples with pre-annotated layout elements as training samples, and the document samples are document samples in the target field; In the layout element recognition model, layout element recognition is performed on the document to obtain a layout element recognition result of the document.
8. The method according to claim 6, characterized in that The step of generating a structure tree of the document according to the recognition result of the layout elements of the document comprises: Determining the position information of the layout elements of the document in the document according to the recognition result of the layout elements of the document; Determining the dependency relationship between the layout elements of the document according to the position information of the layout elements of the document in the document; A structure tree of the document is generated according to the hierarchical relationship between the layout elements of the document and the dependency relationship between the layout elements.
9. The method according to claim 7, characterized in that: The step of generating a structure tree of the document according to the hierarchical relationship between the layout elements of the document and the dependency relationship between the layout elements includes: According to the hierarchical relationship between the layout elements of the document and the dependency relationship between the layout elements, traverse downward from the highest level title to the text paragraphs step by step, and use each traversed layout element as a node of the structure tree to obtain the structure tree of the document.
10. The method according to claim 6, characterized in that The step of segmenting the target domain document to obtain the plurality of initial document segments includes: Determining the body of the document according to the recognition result of the layout elements of the document; Determining a target text length of each of the multiple initial document fragments according to a set text length and structural path information of the text to which each of the multiple initial document fragments belongs; Using a sliding window method, the main body of the document is divided into a plurality of initial document segments whose text length does not exceed the target text length; the size of the sliding window used by the sliding window method is less than or equal to the set text length, and the step size is less than the set text length.
11. The method according to claim 5, characterized in that Before calculating the similarity between the query vector and the pre-stored vectors of the plurality of document fragments, the method further includes: Vectorizing the multiple document fragments using a pre-trained semantic vector model to obtain vectors of the multiple document fragments; the semantic vector model is obtained by model training using the corpus data of the target domain as training samples; The vectorizing the query requirement description information to obtain a query vector corresponding to the query requirement description information includes: The query requirement description information is vectorized using the semantic vector model to obtain a query vector corresponding to the query requirement description information.
12. A method for querying power data, characterized in that: include: Obtain query requirement description information for power data; Determining a target document segment from the power document according to the query requirement description information; Determining target prompt information according to the target metadata type associated with the query requirement description information; The target prompt information is used to describe a processing method for the target metadata type; the target metadata type is at least one metadata of the power document; According to the processing method described in the target prompt information and the query requirement description information, the target document fragment is processed to obtain the query result Wherein, determining the target document segment from the power document according to the query requirement description information includes: Calculating the matching degree between the query requirement description information and a plurality of document fragments corresponding to the power document; the plurality of document fragments are obtained by segmenting the power document; Determine a candidate document segment from the multiple document segments according to the matching degree between the query requirement description information and the multiple document segments; the matching degree between the candidate document segment and the query requirement description information is in the top N positions; N ≥ 2 and is an integer; Performing feature extraction on the candidate document segments respectively to obtain multi-dimensional features of the candidate document segments; Performing feature extraction on the query demand description information to obtain multi-dimensional features of the query demand description information; Calculating the correlation between the candidate document fragment and the query requirement description information according to the multi-dimensional features of the candidate document fragment and the multi-dimensional features of the query requirement description information; The target document segment is determined from the candidate document segments according to the correlation between the candidate document segments and the query requirement description information.
13. An electronic device, characterized in that: include: A memory and a processor; wherein the memory is used to store a computer program; The processor is coupled to the memory and is configured to execute the computer program to perform the steps of the method according to any one of claims 1 to 12.
14. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 1 to 12.
15. A computer program product, characterized in that The method comprises a computer program, which, when executed by one or more processors, causes the one or more processors to execute the steps of the method according to any one of claims 1 to 12.
Citation Information
Patent Citations
Evidence document retrieval method and system based on semantic matching and storage medium
CN112818093A
Structured query language generation method and device, electronic equipment and storage medium
CN118093621A