A multi-interface cooperative natural language data query method and system

Abstract

The application provides a multi-interface cooperative natural language data query method and system, relates to the technical field of natural language processing, and obtains text query data transmitted by a character input interface, voice query data transmitted by a voice input interface and touch instruction data transmitted by a touch selection interface; the voice query data and the touch instruction data are respectively optimized to obtain a multi-interface fusion text set; based on a preset natural language semantic analysis annotation data set, a semantic analysis result is obtained; a unified query instruction is generated through digital information transmission technology; data sources corresponding to each query field information in the semantic analysis result are determined; and an initial query result is obtained according to preset retrieval rules of each data source, so that accurate analysis and cross-interface unified execution of a natural language query intention are realized, and the accuracy, robustness and interactive experience of data query in a complex scene are improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method and system for querying natural language data through multi-interface collaboration. Background Technology

[0002] With the widespread adoption of smart terminal devices and the diversification of human-computer interaction methods, users' demand for initiating natural language queries through various methods such as text input, voice commands, or touch operations in different scenarios is increasing. In complex application environments such as smart offices, smart vehicles, and smart homes, users often use multiple terminal devices such as mobile phones, tablets, and computers simultaneously, and expect the system to seamlessly integrate query requests from different input channels to achieve collaborative responses across devices and interfaces.

[0003] Currently, mainstream solutions attempt to handle query requests from different interfaces by constructing a unified intermediate semantic representation layer. Existing solutions typically convert data from each input channel into preliminary text, then parse it independently using a shared semantic understanding model, and directly forward the parsing results to the corresponding data source to execute the query. However, existing solutions have certain drawbacks. For example, the lack of a collaborative optimization mechanism during the data conversion process of each input channel can easily lead to discrepancies in the semantic representations generated by different channels. The system struggles to determine whether multiple inputs point to the same query intent, resulting in semantic understanding conflicts or inconsistent responses. Furthermore, the integration of multi-source inputs remains at a shallow concatenation level, failing to deeply integrate contextual semantic information, leading to decreased understanding accuracy in complex query scenarios and making it difficult to meet the practical application requirements of high robustness and high collaboration. Summary of the Invention

[0004] The purpose of this application is to provide a multi-interface collaborative natural language data query method and system to solve the problems in the existing technology that are prone to semantic understanding conflicts or inconsistent responses; insufficient understanding accuracy in complex query scenarios; and difficulty in meeting the practical application requirements of high robustness and high collaboration.

[0005] To address the aforementioned technical problems, in a first aspect, this application provides a multi-interface collaborative natural language data query method, comprising:

[0006] Acquire text query data transmitted from the text input interface, voice query data transmitted from the voice input interface, and touch command data transmitted from the touch selection interface;

[0007] The voice query data and the touch command data are optimized separately to obtain voice text data and touch text data. The text query data, the voice text data, and the touch text data are then integrated to obtain a multi-interface fused text set.

[0008] Based on a pre-defined natural language semantic parsing and annotation dataset, semantic parsing is performed on the multi-interface fused text set to obtain semantic parsing results;

[0009] Using digital information transmission technology, the semantic parsing results are sent to each query initiation interface, and abnormal query initiation interfaces in each query initiation interface are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests, and then a unified query instruction is generated.

[0010] Determine the data source corresponding to each query domain information in the semantic parsing results;

[0011] Based on the preset retrieval rules of each data source, the unified query command is matched and retrieved to obtain initial query results, which are then used to perform multi-interface collaborative natural language data query.

[0012] Optionally, the voice query data and the touch command data are optimized separately to obtain voice text data and touch text data. The text query data, the voice text data, and the touch text data are then integrated to obtain a multi-interface fused text set, including:

[0013] Acoustic features are extracted from the voice query data and combined with historical voice samples to construct voice matching relationships. The acoustic features include frequency features, amplitude features, and duration features.

[0014] Based on the acoustic features and the speech matching relationship, the speech query data is divided into multiple text data segments. The pause intervals in the text data segments are semantically connected and spliced ​​together. The ambiguous pronunciations are corrected to obtain speech text data.

[0015] A coordinate sequence is extracted from the touch command data, the coordinate sequence including a start coordinate point, an end coordinate point, and an intermediate coordinate point;

[0016] Based on the preset functional area division table, the functional description corresponding to each coordinate point in the coordinate sequence is determined, each functional description is converted into a corresponding instruction text data segment, duplicate instruction data in the instruction text data segment is deduplicated, and valid instruction data in the conflicting instruction data in the instruction text data segment is filtered according to the preset priority rules to form touch text data.

[0017] Based on the order in which the first timestamp of the text query data, the second timestamp of the voice text data, and the third timestamp of the touch text data are obtained, the text query data, the voice text data, and the touch text data are arranged and interface-marked to obtain a multi-interface fused text set.

[0018] Optionally, based on a pre-defined natural language semantic parsing annotation dataset, semantic parsing is performed on the multi-interface fused text set to obtain semantic parsing results, including:

[0019] Multiple standard semantic items, multiple semantic relationship labels, and multiple domain classification labels are extracted from the pre-defined natural language semantic parsing and annotation dataset;

[0020] Based on the semantic pause markers of natural language, the multi-interface fused text set is divided into multiple semantic segments, and the matching degree between each semantic segment and each standard semantic item is calculated.

[0021] The standard semantic item corresponding to the highest numerical matching degree in each semantic segment is taken as the target semantic item of the corresponding semantic segment. The target semantic item, semantic relationship label and domain classification label of each semantic segment are associated to obtain multiple labeled semantic segments.

[0022] Based on the original data order of the multi-interface fused text set, all tagged semantic fragments are arranged to obtain a semantic fragment set;

[0023] Based on the semantic relationship tags of adjacent tagged semantic fragment pairs in the semantic fragment set, construct the association logic of each tagged semantic fragment in the semantic fragment set;

[0024] The target semantic item, the semantic relationship label, the domain classification label, and the association logic are integrated to form a semantic parsing result.

[0025] Optionally, based on the semantic relationship tags of adjacent tagged semantic fragment pairs in the semantic fragment set, the association logic of each tagged semantic fragment in the semantic fragment set is constructed, including:

[0026] The semantic relationship tags of all adjacent labeled semantic fragment pairs in the semantic fragment set are parsed to obtain the association type of each adjacent labeled semantic fragment pair. The association type includes causal relationship, parallel relationship and progressive relationship.

[0027] Calculate the matching degree between each association type and the preset relationship type rules, and add association type identifiers to adjacent tagged semantic fragment pairs corresponding to association types with matching degrees exceeding the preset matching threshold to form fragment relationship chains;

[0028] Based on the preset association type hierarchy rules, the association types in the fragment relationship chain are hierarchically divided to obtain association types with different priorities;

[0029] The association types with different priorities are compared with the preset priorities to obtain the priority comparison results. Based on the priority comparison results, the adjacent tagged semantic fragment pairs in the fragment relationship chain are rearranged to obtain the adjusted fragment relationship chain.

[0030] Based on the adjusted fragment relationship chain, the association logic of the semantic fragment set is constructed.

[0031] Optionally, the semantic parsing results are sent to each query initiation interface using digital information transmission technology, and abnormal query initiation interfaces are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests. Then, a unified query instruction is generated, including:

[0032] Based on the transmission protocol of each query initiation interface, the semantic parsing results are converted into data to obtain interface-compatible data. The interface-compatible data includes first compatible data corresponding to the text input interface, second compatible data corresponding to the voice input interface, and third compatible data corresponding to the touch selection interface.

[0033] Using digital information transmission technology, the interface-compatible data is sent to the corresponding query initiation interface, and the reception status information of each query initiation interface is recorded. The reception status information includes the reception feedback time and the reception integrity identifier.

[0034] Based on the received status information, an abnormal query initiation interface is identified as having an abnormal received status. The abnormal query initiation interface includes a query initiation interface where the received feedback time exceeds a preset time threshold or a query initiation interface where the received integrity identifier is incomplete.

[0035] Send corresponding interface compatibility data to all abnormal query initiation interfaces to update the reception status information of all abnormal query initiation interfaces until all abnormal query initiation interfaces are updated to normal query initiation interfaces.

[0036] Send an interaction synchronization confirmation request to all normal query initiation interfaces, and after receiving the interaction synchronization confirmation response from all normal query initiation interfaces, extract the query intent corresponding to each normal query initiation interface, perform consistency verification on all query intents, and integrate the query intents that pass the verification into a unified query command.

[0037] Optionally, determining the data source corresponding to each query domain information in the semantic parsing result includes:

[0038] Based on the correspondence between domains and data sources in historical query records, construct a domain-data source correspondence table;

[0039] Multiple query domain information is extracted from the semantic parsing results. Each query domain information is matched with the domain label in the corresponding table of the domain data source to obtain multiple candidate data sources corresponding to each query domain information. At least one candidate data source is extracted from the multiple candidate data sources corresponding to each query domain information to form multiple candidate data source groups.

[0040] Calculate the overall domain coverage of each candidate data source group relative to all query domain information, and mark the candidate data source group with the largest overall domain coverage as the target data source group. If there are multiple target data source groups, retain the target data source group with the fewest candidate data sources.

[0041] The candidate data sources in the target data source group are marked as the data sources corresponding to the query domain information in the semantic parsing results.

[0042] Optionally, based on the preset retrieval rules of each data source, the unified query command is subjected to data matching and retrieval to obtain initial query results, and then multi-interface collaborative natural language data query is performed based on the initial query results, including:

[0043] The search keywords are extracted from the unified query command, and the search keywords are converted into suitable search statements according to the preset search rules of each data source.

[0044] The adapted search statement is sent to the corresponding target data source, and the adapted search statement and the data entries stored in the target data source are matched to obtain multiple matching data entries containing at least one search keyword.

[0045] Calculate the relevance between the matched data entries and the unified query command, and arrange all the matched data entries according to the order of relevance to obtain the arrangement result;

[0046] The first preset number of matching data entries in the sorting results are retained to form the initial query results;

[0047] According to the output format requirements of each query initiation interface, the initial query results are converted to obtain text format output data, voice format output data and touch interaction format output data.

[0048] The text format output data, the voice format output data, and the touch interaction format output data are respectively fed back to the corresponding query initiation interface to complete the natural language data query with multi-interface collaboration.

[0049] Secondly, this application provides a multi-interface collaborative natural language data query system, including:

[0050] The acquisition module is used to acquire text query data transmitted from the text input interface, voice query data transmitted from the voice input interface, and touch command data transmitted from the touch selection interface.

[0051] An optimization module is used to optimize the voice query data and the touch command data respectively to obtain voice text data and touch text data, and to integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fused text set;

[0052] The parsing module is used to perform semantic parsing on the multi-interface fused text set based on a preset natural language semantic parsing annotation dataset, and obtain semantic parsing results;

[0053] The sending module is used to send the semantic parsing results to each query initiation interface through digital information transmission technology, and to process abnormal query initiation interfaces in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests, and then generate a unified query instruction.

[0054] The determination module is used to determine the data source corresponding to each query domain information in the semantic parsing result;

[0055] The retrieval module performs data matching and retrieval on the unified query command according to the preset retrieval rules of each data source to obtain initial query results, and then performs multi-interface collaborative natural language data query based on the initial query results.

[0056] Thirdly, this application provides an electronic device, comprising:

[0057] Memory, used to store computer programs;

[0058] A processor, configured to execute the computer program, implement the steps of a multi-interface collaborative natural language data query method as described in the first aspect above.

[0059] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the multi-interface collaborative natural language data query method described in the first aspect above.

[0060] This application provides a multi-interface collaborative natural language data query method, which acquires text query data transmitted from a text input interface, voice query data transmitted from a voice input interface, and touch command data transmitted from a touch selection interface; optimizes the voice query data and the touch command data respectively to obtain voice-text data and touch-text data; integrates the text query data, the voice-text data, and the touch-text data to obtain a multi-interface fused text set; performs semantic analysis on the multi-interface fused text set based on a preset natural language semantic parsing annotation dataset to obtain semantic analysis results; and sends the semantic analysis results to a digital information transmission technology. Each query initiation interface is configured, and any abnormal query initiation interfaces are processed until all query initiation interfaces are confirmed to be normal and have received interactive synchronization confirmation requests. A unified query command is then generated. The data sources corresponding to the query domain information in the semantic parsing results are determined. Based on the preset retrieval rules of each data source, the unified query command is used for data matching and retrieval to obtain initial query results. Multi-interface collaborative natural language data query is then performed based on these initial results. By acquiring text query data transmitted from the text input interface, voice query data transmitted from the voice input interface, and touch command data transmitted from the touch selection interface, multimodal input source management is achieved. Comprehensive data collection ensures the system can simultaneously respond to user queries initiated through different interaction methods, resolving comprehension biases caused by missing or incorrect information from a single channel. Utilizing standardized semantic items and a tagging system for structured parsing of the fused text improves the system's accuracy in understanding the semantic intent of complex multi-source inputs and its domain recognition capabilities. It ensures the consistency of responses from each channel before query command generation and the overall robustness of the system. It achieves precise targeted retrieval in multi-source heterogeneous data environments, guaranteeing the relevance and completeness of query results, thus completing the entire process of multi-interface collaborative natural language data query. Furthermore, this application extracts data from a pre-defined natural language semantic parsing and annotation dataset. The system uses standard semantic items, semantic relationship tags, and domain classification tags, selecting the standard semantic item with the highest matching degree as the target semantic item for each semantic segment. The target semantic item is then associated with its corresponding semantic relationship tag and domain classification tag to form a labeled semantic segment. Based on the semantic relationship tags between adjacent segments, the system constructs their association logic, integrating the target semantic item, semantic relationship tag, domain classification tag, and association logic to generate a complete semantic parsing result. This approach resolves the inconsistency in understanding caused by semantic expression deviations in different input channels, enhancing the system's ability to deeply understand contextual semantics and improve intent recognition accuracy in complex query scenarios. This, in turn, improves the overall semantic consistency and response reliability of multi-interface collaborative queries. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A flowchart illustrating a multi-interface collaborative natural language data query method provided in this application embodiment;

[0063] Figure 2 A schematic diagram illustrating a specific implementation of a multi-interface collaborative natural language data query method provided in this application embodiment;

[0064] Figure 3 A schematic diagram of the structure of a multi-interface collaborative natural language data query system provided in an embodiment of this application; Detailed Implementation

[0065] To address the need for users to concurrently initiate natural language queries via text, voice, and touch in multi-terminal, multi-interface collaborative scenarios such as smart offices, smart vehicles, and smart homes, this application aims to resolve issues in existing solutions, including semantic representation bias caused by independent data conversion and parsing of each input channel, shallow integration of multi-source information, and inconsistent cross-interface responses. This is achieved by uniformly collecting text query data from text input interfaces, voice query data from voice input interfaces, and touch command data from touch selection interfaces, thus achieving comprehensive coverage of multimodal input. Targeted optimization processing is applied to voice and touch data to improve the semantic expression quality of non-text input, and the three types of data are integrated into a unified multi-interface fusion... By merging text sets, the limitations of independent semantic parsing of each channel in existing solutions are overcome, enhancing the completeness and consistency of input information. Furthermore, a pre-defined natural language semantic parsing annotation dataset is used to perform deep semantic parsing on the merged text sets. The parsing results are fed back to each query initiation interface via digital information transmission technology, ensuring that all interfaces achieve interactive consistency before generating unified query commands, avoiding incorrect command transmission due to interface anomalies or response delays. Based on the parsing results, the corresponding data source is determined and targeted retrieval is performed, enabling efficient collaborative querying of multi-source heterogeneous data. This results in a natural language data query system with more accurate semantic understanding, more reliable interface collaboration, and a smoother user experience.

[0066] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] The core of this application is to provide a multi-interface collaborative natural language data query method, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0068] Step 101: Obtain text query data transmitted from the text input interface, voice query data transmitted from the voice input interface, and touch command data transmitted from the touch selection interface.

[0069] In this step, the text input interface refers to the interface used to receive user-inputted text query requests; text query data refers to user-inputted text query data transmitted through the text input interface; the voice input interface refers to the interface used to receive user-inputted voice query requests; voice query data refers to user-inputted voice query data transmitted through the voice input interface; the touch selection interface refers to the interface used to receive user-inputted touch operation query requests; and touch command data refers to user-inputted touch operation query command data transmitted through the touch selection interface.

[0070] In this embodiment, the text input interface, voice input interface, and touch selection interface are activated to receive information transmitted by the user through these three interfaces. Specifically, the text input interface obtains the text query data entered by the user, the voice input interface obtains the voice query data entered by the user, and the touch selection interface obtains the touch command data generated by the user's touch operation, thus completing the initial acquisition of multi-interface query data.

[0071] Step 102: Optimize the voice query data and the touch command data respectively to obtain voice text data and touch text data. Integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fused text set.

[0072] In this step, voice text data refers to the text data obtained after optimizing voice query data; touch text data refers to the text data obtained after optimizing touch command data; and multi-interface fusion text set refers to the unified text set formed by integrating text query data, voice text data, and touch text data.

[0073] Step 103: Based on the preset natural language semantic parsing annotation dataset, perform semantic parsing on the multi-interface fused text set to obtain the semantic parsing results.

[0074] In this step, the pre-built natural language semantic parsing annotation dataset refers to a pre-constructed annotation dataset containing standard semantic items, semantic relationship labels, and domain classification labels; the semantic parsing result refers to the set of results obtained after performing semantic parsing on the multi-interface fused text set.

[0075] Step 104: Using digital information transmission technology, send the semantic parsing results to each query initiation interface, and process the abnormal query initiation interfaces in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received the interactive synchronization confirmation request, and then generate a unified query instruction.

[0076] In this step, digital information transmission technology refers to the technology used to realize the transmission of data between different interfaces; query initiation interface refers to the interface that initiates a query request; abnormal query initiation interface refers to the query initiation interface that receives semantic parsing results with anomalies; normal query initiation interface refers to the query initiation interface that receives semantic parsing results normally; interactive synchronization confirmation request refers to the request used to confirm whether each query initiation interface has completed the reception of semantic parsing results and interactive synchronization; unified query instruction refers to the unified query instruction formed by integrating the verified query intents corresponding to each query initiation interface.

[0077] Step 105: Determine the data source corresponding to each query domain information in the semantic parsing results.

[0078] In this step, query domain information refers to the knowledge domain category information to which the query request belongs, extracted from the semantic parsing results; data source refers to the source used to provide the data required for the query.

[0079] Step 106: Based on the preset retrieval rules of each data source, perform data matching retrieval on the unified query command to obtain initial query results, and then perform multi-interface collaborative natural language data query based on the initial query results.

[0080] In this step, the preset retrieval rules refer to the rules for data retrieval set in advance by each data source; data matching retrieval refers to the process of searching for data in the data source that matches the unified query command according to the preset retrieval rules; and the initial query results refer to the preliminary query results obtained through data matching retrieval.

[0081] This application embodiment realizes multi-interface collaborative natural language data query, which improves the utilization rate of query data, the accuracy of semantic parsing, the reliability of interface interaction and the accuracy of data retrieval, and meets the query needs of users in multiple scenarios.

[0082] This application provides a specific embodiment. Step 102 involves optimizing the voice query data and the touch command data to obtain voice text data and touch text data. The text query data, the voice text data, and the touch text data are then integrated to obtain a multi-interface fused text set. This specifically includes the following steps:

[0083] Step 201: Extract acoustic features from the voice query data, combine them with historical voice samples, and construct a voice matching relationship. The acoustic features include frequency features, amplitude features, and duration features.

[0084] In this step, acoustic features refer to the features extracted from the speech query data to characterize speech characteristics; historical speech samples refer to previously stored speech data for reference; speech matching relationships refer to the correspondence between acoustic features constructed based on historical speech samples and standard speech units; frequency features refer to features characterizing the frequency attributes of speech signals; amplitude features refer to features characterizing the vibration amplitude attributes of speech signals; and duration features refer to features characterizing the duration attributes of speech signals.

[0085] In this embodiment, the acquired voice query data is analyzed to extract frequency features, amplitude features, and duration features that reflect voice characteristics. These features together constitute acoustic features. The stored historical voice samples are called up. These samples contain a variety of standard voice units and their corresponding acoustic feature data. By comparing the acoustic features of the current voice query data with the acoustic features of the standard voice units in the historical voice samples, a voice matching relationship between the current acoustic features and the standard voice units is established, laying the foundation for the subsequent conversion of voice query data into text data.

[0086] Step 202: Based on the acoustic features and the speech matching relationship, the speech query data is divided into multiple text data segments. The pause intervals in the text data segments are semantically connected and spliced ​​together. The ambiguous pronunciations are corrected to obtain speech text data.

[0087] In this step, the text data segment refers to the fragmented text obtained after dividing the voice query data; the pause interval in the text data segment refers to the discontinuous part of the text data segment caused by speech pauses; and the ambiguous pronunciation part refers to the ambiguous part of the text data segment caused by unclear pronunciation.

[0088] In this embodiment, based on the constructed acoustic features and speech matching relationships, the speech query data is segmented according to the boundaries of speech units and transformed into multiple independent text data segments. For each text data segment, the pause intervals caused by speech pauses are identified, and combined with the contextual semantic logic, connecting content is added to make the part form a coherent expression. At the same time, the ambiguous pronunciation parts caused by unclear pronunciation in the text data segment are identified, and compared with the text corresponding to the standard pronunciation in the same or similar context in historical speech samples, the ambiguous parts are corrected to ensure the accuracy of the text expression, and finally complete and accurate speech text data is obtained.

[0089] Step 203: Extract the coordinate sequence from the touch command data. The coordinate sequence includes the starting coordinate point, the ending coordinate point, and the intermediate coordinate point.

[0090] In this step, the coordinate sequence refers to the set of coordinates representing the touch operation trajectory extracted from the touch instruction data; the starting coordinate point refers to the coordinates corresponding to the starting position of the touch operation in the coordinate sequence; the ending coordinate point refers to the coordinates corresponding to the ending position of the touch operation in the coordinate sequence; and the intermediate coordinate point refers to the coordinates corresponding to the touch position between the starting coordinate point and the ending coordinate point in the coordinate sequence.

[0091] In this embodiment of the application, the touch command data is parsed to extract the coordinate sequence that records the user's touch operation path. The coordinate sequence includes the starting coordinate point when the user starts touching, the ending coordinate point when the user ends touching, and all intermediate coordinate points passed during the touch process, which completely presents the trajectory of the user's touch operation and provides data support for subsequently determining the function corresponding to the touch operation.

[0092] Step 204: Based on the preset functional area division table, determine the functional description corresponding to each coordinate point in the coordinate sequence, convert each functional description into a corresponding instruction text data segment, deduplicate duplicate instruction data in the instruction text data segment, and filter the valid instruction data in the conflicting instruction data in the instruction text data segment according to the preset priority rules to form touch text data.

[0093] In this step, the preset functional area division table refers to a pre-defined table that clarifies the functions corresponding to each area of ​​the touch interface; the function description refers to the function description corresponding to the touch operation determined by the area where the coordinate point is located; the instruction text data segment refers to the instruction fragment in text form obtained after converting the function description; duplicate instruction data refers to instruction data with the same content in the instruction text data segment; the preset priority rule refers to the preset priority standard used to filter conflicting instruction data; conflicting instruction data refers to instruction data with contradictory content in the instruction text data segment; and valid instruction data refers to instruction data that meets the preset priority rule and is selected from the conflicting instruction data.

[0094] In this embodiment, a preset functional area division table is invoked. Based on the coordinate range of each coordinate point in the coordinate sequence, the table is searched to determine the functional description corresponding to each coordinate point. Each functional description is converted into an instruction text data segment that can reflect the meaning of the instruction according to a preset text conversion rule. All instruction text data segments are compared to identify duplicate instruction data with the same content. Redundant duplicate data is deleted through deduplication. If there are conflicting instruction data with contradictory content, valid instruction data that meets the rules is selected according to a preset priority rule. The valid instruction data is then integrated to form touch text data.

[0095] Step 205: Based on the order in which the first timestamp of the text query data, the second timestamp of the voice text data, and the third timestamp of the touch text data are obtained, the text query data, the voice text data, and the touch text data are arranged and interface-marked to obtain a multi-interface fused text set.

[0096] In this step, the first timestamp indicates the time when the text query data is obtained; the second timestamp indicates the time when the voice text data is obtained; and the third timestamp indicates the time when the touch text data is obtained.

[0097] In this embodiment, the time information recorded during data acquisition is retrieved, wherein the first timestamp corresponds to the acquisition time of text query data, the second timestamp corresponds to the acquisition time of voice text data, and the third timestamp corresponds to the acquisition time of touch text data; the text query data, voice text data, and touch text data are sorted according to the order of these three timestamps to ensure that the data is presented in the order of user operation time; at the same time, a corresponding interface source mark is added to each type of data to clarify the source of the data, and finally integrated to form a multi-interface fused text set.

[0098] This application embodiment extracts acoustic features from voice query data and constructs voice matching relationships by combining them with historical voice samples. This process converts the voice query data into accurate voice-text data. It also extracts coordinate sequences from touch command data and processes them into precise touch text data by combining them with a preset functional area division table. Finally, it sorts the three types of data according to timestamps and marks them with interfaces to form a multi-interface fused text set. This solves the problems of heterogeneous data formats, incomplete or inaccurate descriptions in multi-interface query data, and provides a high-quality and unified data foundation for subsequent semantic parsing, thereby improving the efficiency and accuracy of multi-interface collaborative queries.

[0099] This application provides a specific embodiment, such as Figure 2As shown, step 103 involves performing semantic parsing on the multi-interface fused text set based on a pre-set natural language semantic parsing annotation dataset to obtain the semantic parsing results. This specifically includes the following steps:

[0100] Step 301: Extract multiple standard semantic items, multiple semantic relationship labels, and multiple domain classification labels from the pre-set natural language semantic parsing and annotation dataset.

[0101] In this step, the standard semantic item refers to the standard semantic unit used for reference in the pre-defined natural language semantic parsing and annotation dataset; the semantic relationship label refers to the label used to identify the relationship between semantics in the pre-defined natural language semantic parsing and annotation dataset; and the domain classification label refers to the label used to identify the knowledge domain to which the semantic belongs in the pre-defined natural language semantic parsing and annotation dataset.

[0102] In this embodiment of the application, a preset natural language semantic parsing annotation dataset is invoked, and multiple standard semantic items for comparison with subsequent semantic segments are selected from the dataset. These standard semantic items cover the core semantics in common query scenarios. At the same time, multiple semantic relationship tags for describing the association between different semantics and multiple domain classification tags for classifying the knowledge domain to which the semantics belong are extracted, providing a reference for the parsing of subsequent semantic segments.

[0103] Step 302: Based on the semantic pause markers of natural language, divide the multi-interface fused text set into multiple semantic segments, and calculate the matching degree between each semantic segment and each standard semantic item.

[0104] In this step, semantic pause markers in natural language refer to pause markers used to distinguish different semantic units in natural language, such as commas and periods; semantic segments refer to independent semantic units obtained by dividing the multi-interface fused text set according to semantic pause markers; and matching degree refers to the quantitative value of the similarity between semantic segments and standard semantic items.

[0105] In this embodiment of the application, text analysis is performed on the multi-interface fused text set to identify semantic pause markers in the natural language. Using these markers as dividing boundaries, the multi-interface fused text set is split into multiple semantic segments that can independently express complete semantics. For each semantic segment, the keywords it contains are compared with the keywords contained in each standard semantic item, and the matching degree between the two is calculated as (the number of overlapping keywords between the semantic segment and the standard semantic item / the total number of keywords in the standard semantic item) × 100%, thus obtaining the matching degree value between each semantic segment and each standard semantic item.

[0106] Step 303: Take the standard semantic item corresponding to the largest numerical matching degree in each semantic segment as the target semantic item of the corresponding semantic segment, and associate the target semantic item, semantic relationship label and domain classification label of each semantic segment to obtain multiple labeled semantic segments.

[0107] In this step, the target semantic item refers to the standard semantic item with the highest matching degree in the semantic fragment; the labeled semantic fragment refers to the semantic fragment that is associated with the target semantic item, the semantic relationship label, and the domain classification label.

[0108] In this embodiment, multiple matching degree values ​​corresponding to each semantic segment are compared, and the matching degree with the largest value is selected. The standard semantic item corresponding to the largest matching degree is determined as the target semantic item of the semantic segment. Based on the association information of the target semantic item in the preset natural language semantic parsing annotation dataset, the semantic relationship label and domain classification label corresponding to the target semantic item are found. The target semantic item, the corresponding semantic relationship label and the domain classification label are bound to the original semantic segment, so that each semantic segment has a clear semantic identifier, forming multiple labeled semantic segments.

[0109] Step 304: Arrange all tagged semantic fragments according to the original data order of the multi-interface fused text set to obtain a semantic fragment set.

[0110] In this step, the original data order refers to the original arrangement order of each data in the multi-interface fused text set; the semantic fragment set refers to the set formed after arranging all labeled semantic fragments in the original data order.

[0111] In this embodiment of the application, the original data order recorded during the integration of the multi-interface fusion text set is retrieved. This order reflects the sequential logic of user operations and the original association of data. According to this original data order, all previously obtained tagged semantic fragments are arranged sequentially to ensure that the order of the tagged semantic fragments is consistent with the user operation logic and the original association of data, so as to avoid semantic logic breakage due to disordered order. After the arrangement is completed, a semantic fragment set is formed.

[0112] Step 305: Based on the semantic relationship tags of adjacent tagged semantic fragment pairs in the semantic fragment set, construct the association logic of each tagged semantic fragment in the semantic fragment set.

[0113] In this step, an adjacent labeled semantic fragment pair refers to a pair of two labeled semantic fragments that are adjacent in position within the semantic fragment set; the association logic refers to the logical relationship used to describe the semantic association between the labeled semantic fragments in the semantic fragment set.

[0114] In this embodiment, the semantic fragment set is traversed to identify all adjacent labeled semantic fragments. Each pair of adjacent labeled semantic fragments forms an adjacent labeled semantic fragment pair. The semantic relationship tags of the two labeled semantic fragments in each adjacent labeled semantic fragment pair are extracted. By analyzing the meaning and interrelationship of these two semantic relationship tags, the semantic association mode between adjacent labeled semantic fragment pairs is determined. Based on the semantic association mode of all adjacent labeled semantic fragment pairs, the overall semantic connection between all labeled semantic fragments in the semantic fragment set is sorted out, and an association logic that can reflect the logical relationship of each labeled semantic fragment is constructed.

[0115] Step 306: Integrate the target semantic item, the semantic relationship label, the domain classification label, and the association logic to form a semantic parsing result.

[0116] In this embodiment of the application, the target semantic items, semantic relationship tags, and domain classification tags corresponding to all labeled semantic fragments are collected, and this information is integrated with the constructed association logic; ensuring that each target semantic item can be associated with other target semantic items through semantic relationship tags, forming a semantic parsing result containing complete semantic information and logical association.

[0117] The embodiments of this application solve the problems of semantic fragmentation and unclear association in multi-interface fused text sets, providing a clear and structured semantic basis for the subsequent generation of unified query instructions, and improving the semantic understanding accuracy of multi-interface collaborative queries.

[0118] For example, the pre-defined natural language semantic parsing and annotation dataset contains standard semantic items such as querying device parameters and querying device installation guides, semantic relationship labels such as parallel, progressive, and independent relationships, and domain classification labels for device information querying. These standard semantic items, semantic relationship labels, and domain classification labels are extracted from this dataset. The previously formed multi-interface fusion text set consists of: [Text Input Interface] Model B Device Parameters; [Voice Input Interface] Querying Installation Guide for Model B Devices; [Touch Selection Interface] Querying Installation Guide for Model B Devices. Based on the semicolons, it is divided into three semantic segments: [Text Input Interface] Model B Device Parameters, [Voice Input Interface] Querying Installation Guide for Model B Devices, and [Touch Selection Interface] Querying Installation Guide for Model B Devices.

[0119] The matching degree between each semantic fragment and each standard semantic item is calculated as follows: (Number of overlapping keywords between the semantic fragment and the standard semantic item / Total number of keywords in the standard semantic item) × 100%. For the first semantic fragment [text input interface] B model equipment parameters, its keywords are extracted as B model equipment and parameters; the keywords of the standard semantic item for equipment parameters are equipment and parameters, with 2 overlapping keywords and a total of 2 keywords, so the matching degree is calculated as (2 / 2) × 100% = 100%; the keywords of the standard semantic item for equipment installation guide are equipment and installation guide, with 1 overlapping keyword and a total of 2 keywords, so the matching degree is calculated as (1 / 2) × 100% = 50%.

[0120] For the second semantic segment [voice input interface] querying the installation guide for model B equipment, the extracted keywords are query, model B equipment, installation guide; the keywords for the standard semantic item of querying the installation guide are query, equipment, installation guide, with 3 overlapping keywords and a total of 3 keywords, the matching degree is calculated as (3 / 3) × 100% = 100%; the keywords for the standard semantic item of querying the parameters are query, equipment, parameters, with 2 overlapping keywords and a total of 3 keywords, the matching degree is calculated as (2 / 3) × 100% ≈ 66.7%.

[0121] The third semantic segment [touch selection interface] queries the installation guide for model B equipment. The keywords extracted are query, model B equipment, and installation guide. The keywords for the standard semantic item of the equipment installation guide are query, equipment, and installation guide. Query, equipment, and installation guide overlap with the keywords of the third semantic segment, with 3 overlapping keywords. The total number of keywords for the standard semantic item of the equipment installation guide is 3. The matching degree is calculated as (3 / 3) × 100% = 100%. The keywords for the standard semantic item of the equipment parameters are query, equipment, and parameters. Query, equipment, and parameters overlap with the keywords of the third semantic segment, with 2 overlapping keywords. The total number of keywords for the standard semantic item of the equipment parameters is 3. The matching degree is calculated as (2 / 3) × 100% ≈ 66.7%.

[0122] The target semantic item for the first semantic fragment is determined to be the query device parameters. Based on a pre-defined dataset, it is associated with device information query domain tags and independent relationship tags. The target semantic items for the second and third semantic fragments are determined to be the query device installation guide. These are associated with device information query domain tags and parallel relationship tags, forming three labeled semantic fragments. Following the original data order of the multi-interface fused text set, the three labeled semantic fragments are arranged into a semantic fragment set. Adjacent labeled semantic fragment pairs within the set are identified. The semantic relationship tags for the first and second fragment pairs are independent and parallel, respectively. Analysis shows that both semantically revolve around querying device model B, indicating a parallel relationship. The semantic relationship tags for the second and third fragment pairs are both parallel, confirming a parallel relationship. Based on these associations, a connection logic is constructed where all three fragments belong to the device information query domain and are parallel to each other. The target semantic item, semantic relationship tags, domain classification tags, and connection logic are integrated to form the semantic parsing result.

[0123] This application provides a specific embodiment. Step 305 involves constructing the association logic for each tagged semantic segment in the semantic segment set based on the semantic relationship tags of adjacent tagged semantic segment pairs in the semantic segment set. This specifically includes the following steps:

[0124] Step 311: Parse the semantic relationship tags of all adjacent labeled semantic fragment pairs in the semantic fragment set to obtain the association type of each adjacent labeled semantic fragment pair. The association type includes causal relationship, parallel relationship and progressive relationship.

[0125] In this step, the association type refers to the category used to describe the semantic relationship between adjacent labeled semantic fragment pairs; the causal relationship refers to the relationship in the association type that indicates that one semantic fragment is the cause or result of another semantic fragment; the parallel relationship refers to the relationship in the association type that indicates that the content of multiple semantic fragments is parallel and has no sequential or subordinate relationship; the progressive relationship refers to the relationship in the association type that indicates that the content of semantic fragments deepens layer by layer.

[0126] In this embodiment of the application, the semantic fragment set is traversed, and all adjacent labeled semantic fragments are selected to form multiple adjacent labeled semantic fragment pairs. The semantic relationship tags of the two labeled semantic fragments in each adjacent labeled semantic fragment pair are extracted, and the semantic logic expressed by the two semantic relationship tags is analyzed. The association type corresponding to each adjacent labeled semantic fragment pair is determined from the causal relationship, parallel relationship and progressive relationship, so as to obtain the association type of each adjacent labeled semantic fragment pair.

[0127] Step 312: Calculate the matching degree between each association type and the preset relationship type rule, and add association type identifiers to adjacent tagged semantic fragment pairs corresponding to association types with matching degrees exceeding the preset matching threshold to form fragment relationship chains.

[0128] In this step, the preset relation type rule refers to the pre-defined rule used to determine the validity of the association type; the preset matching threshold refers to the pre-set matching degree threshold used to filter valid association types; the association type identifier refers to the identifier used to mark the association type of adjacent labeled semantic fragment pairs; and the fragment relationship chain refers to the chain formed by sequentially connecting adjacent labeled semantic fragment pairs after adding the association type identifier.

[0129] In this embodiment, a preset relationship type rule is invoked, which clarifies the judgment criteria corresponding to each association type. For each association type, its features are compared with the judgment criteria of the preset relationship type rule, and the matching degree is calculated as (number of overlapping items between the association type features and the preset rule judgment criteria / total number of items in the preset rule judgment criteria) × 100%. The calculated matching degree is compared with a preset matching threshold, and association types with matching degrees exceeding the preset matching threshold are retained. For these association types, an association type identifier that can reflect their association type is added to the adjacent labeled semantic fragment pairs corresponding to these association types. Then, these labeled adjacent labeled semantic fragment pairs are connected sequentially according to the original order of the semantic fragment set to form a fragment relationship chain.

[0130] Step 313: Based on the preset association type hierarchy rules, the association types in the fragment relationship chain are hierarchically divided to obtain association types with different priorities.

[0131] In this step, the preset association type hierarchy rule refers to the pre-defined rules used to divide the priority order of association types; association types with different priorities refer to association types with different priority levels after being divided according to the preset association type hierarchy rule.

[0132] In this embodiment of the application, a preset association type hierarchy rule is retrieved, which clarifies the priority order of association types. According to the rule, all association types in the fragment relationship chain are classified into levels. Association types that conform to the causal relationship rule are classified as the highest priority, those that conform to the progressive relationship rule are classified as the middle priority, and those that conform to the parallel relationship rule are classified as the lowest priority, thus obtaining association types with different priorities and clarifying the priority order of each association type in the fragment relationship chain.

[0133] Step 314: Compare the association types with different priorities with the preset priorities to obtain the priority comparison results. Based on the priority comparison results, rearrange the adjacent tagged semantic fragment pairs in the fragment relationship chain to obtain the adjusted fragment relationship chain.

[0134] In this step, the preset priority refers to the pre-set priority standard for association types used for comparison; the priority comparison result refers to the result obtained after comparing association types with different priorities with the preset priority; and the adjusted fragment relationship chain refers to the chain formed after rearranging the original fragment relationship chain according to the priority comparison result.

[0135] In this embodiment, a preset priority is invoked, and the actual priority of each association type in the fragment relationship chain is compared with the preset priority one by one to determine whether the actual priority is consistent with the preset priority. If they are consistent, they are marked as consistent; if they are inconsistent, they are marked as inconsistent, and the priority comparison result is obtained. For the adjacent labeled semantic fragment pairs corresponding to the association types that do not conform to the priority comparison result, their positions in the fragment relationship chain are adjusted according to the preset priority, and the adjusted fragment relationship chain is obtained after the rearrangement is completed.

[0136] Step 315: Construct the association logic of the semantic fragment set based on the adjusted fragment relationship chain.

[0137] In this embodiment of the application, the association type identifier and arrangement order of each adjacent tagged semantic segment pair in the adjusted segment relationship chain are analyzed, and the semantic association path between all tagged semantic segments is sorted out. Based on the association path, the logical order and association method of each tagged semantic segment are clarified, and these logical relationships are integrated to form an association logic that can fully reflect the semantic connection between all tagged semantic segments in the semantic segment set.

[0138] This application's embodiments solve the problems of chaotic association types and unclear priorities among multi-interface semantic fragments, ensuring the accuracy and orderliness of semantic association logic, and providing clear logical support for the efficient application of subsequent semantic parsing results.

[0139] This application provides a specific embodiment. Step 104 involves sending the semantic parsing results to each query initiation interface using digital information transmission technology, and processing any abnormal query initiation interfaces until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests. A unified query instruction is then generated, specifically including the following steps:

[0140] Step 401: Based on the transmission protocol of each query initiation interface, the semantic parsing result is converted into data to obtain interface-compatible data. The interface-compatible data includes first compatible data corresponding to the text input interface, second compatible data corresponding to the voice input interface, and third compatible data corresponding to the touch selection interface.

[0141] In this step, the transmission protocol refers to the rule standard used for data transmission by each query initiation interface; data conversion refers to the operation of converting the semantic parsing results into a format that conforms to the transmission protocol of each query initiation interface; interface-compatible data refers to data that can be recognized and received by each query initiation interface after data conversion; first compatible data refers to interface-compatible data that matches the transmission protocol of the text input interface; second compatible data refers to interface-compatible data that matches the transmission protocol of the voice input interface; and third compatible data refers to interface-compatible data that matches the transmission protocol of the touch selection interface.

[0142] In this embodiment, the transmission protocol corresponding to each query initiation interface is first determined, and the transmission protocol formats of different interfaces are different. According to the transmission protocol requirements of each interface, the semantic parsing results are converted into data, and the target semantic items, semantic relationship tags and other information in the semantic parsing results are converted into first compatible data that can be recognized by the text input interface, second compatible data that can be recognized by the voice input interface, and third compatible data that can be recognized by the touch selection interface. Finally, interface compatible data containing the first compatible data, the second compatible data and the third compatible data are obtained.

[0143] Step 402: Using digital information transmission technology, send the interface-compatible data to the corresponding query initiation interface and record the reception status information of each query initiation interface. The reception status information includes the reception feedback time and reception integrity identifier.

[0144] In this step, the receiving status information refers to the information recording the interface compatibility status of each query initiation interface; the receiving feedback time refers to the time it takes for the query initiation interface to send back the receiving result after receiving the interface compatibility data; and the receiving integrity identifier refers to the identifier used to identify whether the interface compatibility data received by the query initiation interface is complete.

[0145] In this embodiment, digital information transmission technology is used to send first compatible data to the text input interface, second compatible data to the voice input interface, and third compatible data to the touch selection interface. Simultaneously, a timing function is activated to record the time from when each query initiating interface receives the compatible data to when it responds with a received signal; this time is the reception feedback time. At the same time, a data integrity verification command is sent to each query initiating interface. Each interface performs integrity checks on the received compatible data according to the verification command, generating a complete or incomplete reception integrity identifier. The reception feedback time and reception integrity identifier of each query initiating interface are integrated to form the reception status information of each query initiating interface.

[0146] Step 403: Based on the received status information, identify the abnormal query initiation interface where the received status information is in an abnormal received state. The abnormal query initiation interface includes a query initiation interface where the received feedback time exceeds a preset time threshold or a query initiation interface where the received integrity identifier is incomplete.

[0147] In this step, "abnormal reception status" refers to a state where the data received by the query initiation interface has a problem; "preset time threshold" refers to a pre-set time threshold for judging whether the reception feedback time is abnormal; and "abnormal query initiation interface" refers to a query initiation interface that is in an abnormal reception status.

[0148] In this embodiment, a preset time threshold is retrieved to analyze the reception status information of each query initiation interface. If the reception feedback time of a query initiation interface exceeds the preset time threshold, it indicates that there is a delay in the data received by the interface, and it is determined to be in an abnormal reception state. If the reception integrity flag of a query initiation interface is incomplete, it indicates that there are missing data received by the interface, and it is also determined to be in an abnormal reception state. All query initiation interfaces in an abnormal reception state are filtered out to form an abnormal query initiation interface list, and the abnormal interfaces that need to be processed are identified.

[0149] Step 404: Send corresponding interface compatibility data to all abnormal query initiation interfaces to update the reception status information of all abnormal query initiation interfaces until all abnormal query initiation interfaces are updated to normal query initiation interfaces.

[0150] In this embodiment, compatible data corresponding to the abnormal query initiation interface is selected from the interface compatibility data; the corresponding compatible data is resent to the abnormal query initiation interface using digital information transmission technology; after the transmission is completed, the reception feedback time and reception integrity identifier of the abnormal query initiation interface are re-recorded, and its reception status information is updated; the updated reception status information is checked, and if there are still cases where the reception feedback time exceeds the preset time threshold or the reception integrity identifier is incomplete, the above resending and status update operations are repeated until the reception feedback time of all abnormal query initiation interfaces does not exceed the preset time threshold and the reception integrity identifier is complete, that is, all abnormal query initiation interfaces are updated to normal query initiation interfaces.

[0151] Step 405: Send an interaction synchronization confirmation request to all normal query initiation interfaces, and after receiving the interaction synchronization confirmation response from all normal query initiation interfaces, extract the query intent corresponding to each normal query initiation interface, perform consistency verification on all query intents, and integrate the query intents that pass the verification into a unified query command.

[0152] In this step, the interactive synchronization confirmation response refers to the confirmation signal returned by the normal query initiation interface after receiving the interactive synchronization confirmation request; the query intent refers to the user's query purpose corresponding to the query initiation interface.

[0153] In this embodiment, an interaction synchronization confirmation request is sent to all normal query initiation interfaces. This request is used to confirm whether each interface has accurately received and understood the semantic parsing results. After receiving the request, each normal query initiation interface, if it has accurately received and understood the results, will send back a synchronization confirmation response. After receiving the synchronization confirmation responses from all normal query initiation interfaces, the user's query purpose is extracted from the semantic parsing results corresponding to each normal query initiation interface to obtain the query intent corresponding to each normal query initiation interface. All query intents are checked for consistency to determine whether the query intents of different interfaces revolve around the same core requirement. If they are consistent, the check is considered successful. All successfully checked query intents are integrated, duplicate content is removed, and a unified query instruction that can cover the query requirements of all interfaces is formed.

[0154] This application's embodiments solve the problems of incompatibility in data transmission across multiple interfaces, abnormal reception, and inconsistent query intents, ensuring reliable transmission and synchronization of semantic parsing results across multiple interfaces, and providing a unified instruction basis for subsequent accurate data queries.

[0155] This application provides a specific embodiment. Step 105, determining the data source corresponding to each query domain information in the semantic parsing result, specifically includes the following steps:

[0156] Step 501: Based on the correspondence between domains and data sources in historical query records, construct a domain-data source correspondence table.

[0157] In this step, historical query records refer to previously stored records containing information about the query domain and its corresponding data source; the domain data source mapping table refers to a table built based on historical query records to clarify the correspondence between domains and data sources; and domain tags refer to tags in the domain data source mapping table used to identify different query domains.

[0158] In this embodiment, stored historical query records are retrieved, and the corresponding data is organized, categorized by query domain, and a unique domain label is set for each query domain. The domain label, the corresponding query domain name, and all data source information under that domain are arranged in rows to form a domain data source correspondence table, ensuring that the corresponding data source can be quickly located through the domain label.

[0159] Step 502: Extract multiple query domain information from the semantic parsing results, match each query domain information with the domain label in the corresponding table of the domain data source to obtain multiple candidate data sources corresponding to each query domain information, and extract at least one candidate data source from the multiple candidate data sources corresponding to each query domain information to form multiple candidate data source groups.

[0160] In this step, a candidate data source refers to a potential data source that matches the query domain information; a candidate data source group refers to a set formed by extracting at least one candidate data source from each candidate data source corresponding to each query domain information and combining them.

[0161] In this embodiment, domain information is extracted from the semantic parsing results to identify multiple query domain information contained therein; each query domain information is compared with the domain label in the corresponding table of the domain data source to find the domain label that matches the query domain information, and then all data sources under the domain label are obtained. These data sources are multiple candidate data sources corresponding to each query domain information; at least one candidate data source is selected from the multiple candidate data sources corresponding to each query domain information, and the selected candidate data sources are combined to form multiple candidate data source groups.

[0162] Step 503: Calculate the overall domain coverage of each candidate data source group relative to all query domain information, and mark the candidate data source group with the largest overall domain coverage as the target data source group. If there are multiple target data source groups, retain the target data source group with the fewest candidate data sources.

[0163] In this step, overall domain coverage refers to the extent to which the candidate data source group can cover all query domain information; the target data source group refers to the candidate data source group with the largest overall domain coverage value and the fewest number of candidate data sources.

[0164] In this embodiment, for each candidate data source group, the number of query domain information that can be covered by all candidate data sources in the group is counted; the overall domain coverage is calculated as (number of query domain information covered by the candidate data source group / total number of query domain information) × 100%; the overall domain coverage of all candidate data source groups is compared, and the candidate data source group with the largest overall domain coverage is initially marked as the target data source group; if there are multiple candidate data source groups with the same and the largest overall domain coverage, the number of candidate data sources in these candidate data source groups is counted, and the candidate data source group with the fewest number is retained as the final target data source group.

[0165] Step 504: Mark the candidate data sources in the target data source group as the data sources corresponding to each query domain information in the semantic parsing results.

[0166] In this embodiment of the application, after determining the target data source group, all candidate data sources in the group are extracted; each candidate data source is associated with the corresponding query domain information; through the association operation, the specific data source corresponding to each query domain information is clarified, and finally the candidate data sources in the target data source group are determined as the data sources corresponding to each query domain information in the semantic parsing results.

[0167] This application's embodiments solve the problems of chaotic and incomplete data source matching in multi-query domains, ensuring that the selected data source can accurately cover all query needs, and providing efficient and adaptable data source support for data retrieval of subsequent unified query commands.

[0168] This application provides a specific embodiment. Step 106 involves performing data matching and retrieval on the unified query command according to the preset retrieval rules of each data source to obtain initial query results. This initial query results are then used to perform multi-interface collaborative natural language data querying. Specifically, this includes the following steps:

[0169] Step 601: Extract search keywords from the unified query command, and convert the search keywords into suitable search statements according to the preset search rules of each data source.

[0170] In this step, the search keywords refer to the core terms extracted from the unified query command for data retrieval; the adapted search statement refers to the search statement that can be recognized by the target data source after being transformed according to the preset search rules of each data source; and the target data source refers to the data source that has been determined and corresponds to the information in each query domain.

[0171] In this embodiment, the unified query command is segmented into words to filter out words that can reflect the core query requirements; these words are the search keywords. The preset search rules of each target data source are retrieved. According to the preset search rules of different target data sources, the search keywords are converted into adapted search statements that conform to the search syntax of the data source, ensuring that the adapted search statements can be recognized by the corresponding target data source.

[0172] Step 602: Send the adapted search statement to the corresponding target data source, match the adapted search statement with the data entries stored in the target data source, and obtain multiple matching data entries containing at least one search keyword.

[0173] In this step, a data entry refers to a single data record stored in the target data source; a matching data entry refers to a data entry in the target data source that matches the appropriate search statement and contains at least one search keyword.

[0174] In this embodiment of the application, each adapted search statement is sent to the corresponding target data source according to the correspondence between the target data source and the adapted search statement. After receiving the adapted search statement, the target data source calls the internal search module to compare the content of the adapted search statement with all data entries stored in the data source. Data entries containing at least one search keyword are selected, and these selected data entries are the matching data entries.

[0175] Step 603: Calculate the relevance between the matched data entries and the unified query command, and arrange all the matched data entries according to the order of relevance to obtain the arrangement result.

[0176] In this step, relevance refers to the quantitative value of the degree of association between the matched data entries and the unified query command; high-low order refers to the order in which the relevance values ​​are arranged from largest to smallest; and the sorting result refers to the result formed by arranging all the matched data entries in order of high-low relevance.

[0177] In this embodiment, for each matching data entry, its contained keywords are extracted and compared with the search keywords of the unified query command. The relevance is calculated as (number of overlapping keywords between the matching data entry and the unified query command / total number of search keywords of the unified query command) × 100%. The relevance values ​​of all matching data entries are compared and sorted in descending order of value. After sorting, the arrangement result is obtained to ensure that the matching data entries with higher relevance are placed in the first position.

[0178] Step 604: Retain the first preset number of matching data entries in the sorting results to form the initial query results.

[0179] In this step, the pre-set number refers to the number of matching data entries that need to be retained; the initial query result refers to the result formed after retaining the pre-set number of matching data entries in the sorted result.

[0180] In this embodiment, a pre-set number of matching data entries is retrieved; the sorting results are viewed, and starting from the first matching data entry in the sorting results, a pre-set number of matching data entries are selected sequentially; the selected matching data entries are integrated to remove duplicate information, ensuring the uniqueness and conciseness of the information, and finally forming the initial query results.

[0181] Step 605: Based on the output format requirements of each query initiation interface, the initial query results are converted to obtain text format output data, voice format output data, and touch interaction format output data.

[0182] In this step, the output format requirements refer to the specific requirements of each query initiation interface for the output data format; text format output data refers to data that conforms to the output format requirements of the text input interface; voice format output data refers to data that conforms to the output format requirements of the voice input interface; and touch interaction format output data refers to data that conforms to the output format requirements of the touch selection interface.

[0183] In this embodiment of the application, the output format requirements of each query initiation interface are determined; according to these output format requirements, the initial query results are format converted, the initial query results are converted into text format output data according to text format, the audio + text subtitle format is converted into voice format output data according to audio + text subtitle format, and the clickable interactive card format is converted into touch interactive format output data, so as to ensure that the three types of data after conversion can be recognized and output by the corresponding query initiation interface respectively.

[0184] Step 606: Feed back the text format output data, the voice format output data, and the touch interaction format output data to the corresponding query initiation interface to complete the multi-interface collaborative natural language data query.

[0185] In this embodiment, based on the correspondence between the query initiation interface and the output data, text-formatted output data is fed back to the text input interface, voice-formatted output data is fed back to the voice input interface, and touch-interaction-formatted output data is fed back to the touch selection interface. After receiving the corresponding output data, each query initiation interface presents it to the user according to its own display or playback method. The user can obtain the initial query results through each interface, thereby completing the multi-interface collaborative natural language data query.

[0186] This application's embodiments solve the problems of poor adaptability of multi-data source retrieval, low relevance of query results, and incompatibility of multi-interface output formats, ensuring the accuracy of data retrieval and the adaptability of multi-interface output, and improving the user experience of obtaining query results through multiple interfaces.

[0187] Figure 3 This is a schematic diagram illustrating a specific implementation of a multi-interface collaborative natural language data query system provided in this application. (Refer to...) Figure 3 The system may include:

[0188] The acquisition module 21 is used to acquire text query data transmitted by the text input interface, voice query data transmitted by the voice input interface, and touch command data transmitted by the touch selection interface.

[0189] Optimization module 22 is used to optimize the voice query data and the touch command data respectively to obtain voice text data and touch text data, and integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fused text set;

[0190] Parsing module 23 is used to perform semantic parsing on the multi-interface fused text set based on a preset natural language semantic parsing annotation dataset, and obtain semantic parsing results;

[0191] The sending module 24 is used to send the semantic parsing results to each query initiation interface through digital information transmission technology, and to process abnormal query initiation interfaces in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests, and then generate a unified query instruction.

[0192] The determination module 25 is used to determine the data source corresponding to each query domain information in the semantic parsing result;

[0193] The retrieval module 26 performs data matching and retrieval on the unified query command according to the preset retrieval rules of each data source to obtain initial query results, and then performs multi-interface collaborative natural language data query based on the initial query results.

[0194] This application provides an embodiment of a multi-interface collaborative natural language data query system for implementing the aforementioned multi-interface collaborative natural language data query method. Therefore, the specific implementation of the multi-interface collaborative natural language data query system can be found in the embodiment section of the multi-interface collaborative natural language data query method described above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0195] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described multi-interface collaborative natural language data query methods.

[0196] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described multi-interface collaborative natural language data query methods.

[0197] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0198] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the multi-interface collaborative natural language data query method.

[0199] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0200] The foregoing has provided a detailed description of the multi-interface collaborative natural language data query method and system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A multi-interface collaborative natural language data query method, characterized in that, include: Acquire text query data transmitted from the text input interface, voice query data transmitted from the voice input interface, and touch command data transmitted from the touch selection interface; The voice query data and the touch command data are optimized separately to obtain voice text data and touch text data. The text query data, the voice text data, and the touch text data are then integrated to obtain a multi-interface fused text set. Based on a pre-defined natural language semantic parsing and annotation dataset, semantic parsing is performed on the multi-interface fused text set to obtain semantic parsing results; Using digital information transmission technology, the semantic parsing results are sent to each query initiation interface, and abnormal query initiation interfaces in each query initiation interface are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests, and then a unified query instruction is generated. Determine the data source corresponding to each query domain information in the semantic parsing results; According to the preset retrieval rules of each data source, the unified query command is matched and retrieved to obtain the initial query results, so as to perform multi-interface collaborative natural language data query based on the initial query results; The voice query data and the touch command data are optimized separately to obtain voice text data and touch text data. The text query data, the voice text data, and the touch text data are then integrated to obtain a multi-interface fused text set, including: Acoustic features are extracted from the voice query data and combined with historical voice samples to construct voice matching relationships. The acoustic features include frequency features, amplitude features, and duration features. Based on the acoustic features and the speech matching relationship, the speech query data is divided into multiple text data segments. The pause intervals in the text data segments are semantically connected and spliced ​​together. The ambiguous pronunciations are corrected to obtain speech text data. A coordinate sequence is extracted from the touch command data, the coordinate sequence including a start coordinate point, an end coordinate point, and an intermediate coordinate point; Based on the preset functional area division table, the functional description corresponding to each coordinate point in the coordinate sequence is determined, each functional description is converted into a corresponding instruction text data segment, duplicate instruction data in the instruction text data segment is deduplicated, and valid instruction data in the conflicting instruction data in the instruction text data segment is filtered according to the preset priority rules to form touch text data. Based on the order in which the first timestamp of the text query data, the second timestamp of the voice text data, and the third timestamp of the touch text data are obtained, the text query data, the voice text data, and the touch text data are arranged and interface-marked to obtain a multi-interface fused text set. Based on a pre-defined natural language semantic parsing and annotation dataset, semantic parsing is performed on the multi-interface fused text set to obtain semantic parsing results, including: Multiple standard semantic items, multiple semantic relationship labels, and multiple domain classification labels are extracted from the pre-defined natural language semantic parsing and annotation dataset; Based on the semantic pause markers of natural language, the multi-interface fused text set is divided into multiple semantic segments, and the matching degree between each semantic segment and each standard semantic item is calculated. The standard semantic item corresponding to the highest numerical matching degree in each semantic segment is taken as the target semantic item of the corresponding semantic segment. The target semantic item, semantic relationship label and domain classification label of each semantic segment are associated to obtain multiple labeled semantic segments. Based on the original data order of the multi-interface fused text set, all tagged semantic fragments are arranged to obtain a semantic fragment set; Based on the semantic relationship tags of adjacent tagged semantic fragment pairs in the semantic fragment set, construct the association logic of each tagged semantic fragment in the semantic fragment set; The target semantic item, the semantic relationship label, the domain classification label, and the association logic are integrated to form a semantic parsing result.

2. The method according to claim 1, characterized in that, Based on the semantic relationship tags of adjacent tagged semantic fragment pairs in the semantic fragment set, the association logic of each tagged semantic fragment in the semantic fragment set is constructed, including: The semantic relationship tags of all adjacent labeled semantic fragment pairs in the semantic fragment set are parsed to obtain the association type of each adjacent labeled semantic fragment pair. The association type includes causal relationship, parallel relationship and progressive relationship. Calculate the matching degree between each association type and the preset relationship type rules, and add association type identifiers to adjacent tagged semantic fragment pairs corresponding to association types with matching degrees exceeding the preset matching threshold to form fragment relationship chains; Based on the preset association type hierarchy rules, the association types in the fragment relationship chain are hierarchically divided to obtain association types with different priorities; The association types with different priorities are compared with the preset priorities to obtain the priority comparison results. Based on the priority comparison results, the adjacent tagged semantic fragment pairs in the fragment relationship chain are rearranged to obtain the adjusted fragment relationship chain. Based on the adjusted fragment relationship chain, the association logic of the semantic fragment set is constructed.

3. The method according to claim 1, characterized in that, Using digital information transmission technology, the semantic parsing results are sent to each query initiation interface, and abnormal query initiation interfaces are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests. Then, a unified query instruction is generated, including: Based on the transmission protocol of each query initiation interface, the semantic parsing results are converted into data to obtain interface-compatible data. The interface-compatible data includes first compatible data corresponding to the text input interface, second compatible data corresponding to the voice input interface, and third compatible data corresponding to the touch selection interface. Using digital information transmission technology, the interface-compatible data is sent to the corresponding query initiation interface, and the reception status information of each query initiation interface is recorded. The reception status information includes the reception feedback time and the reception integrity identifier. Based on the received status information, an abnormal query initiation interface is identified as having an abnormal received status. The abnormal query initiation interface includes a query initiation interface where the received feedback time exceeds a preset time threshold or a query initiation interface where the received integrity identifier is incomplete. Send corresponding interface compatibility data to all abnormal query initiation interfaces to update the reception status information of all abnormal query initiation interfaces until all abnormal query initiation interfaces are updated to normal query initiation interfaces. Send an interaction synchronization confirmation request to all normal query initiation interfaces, and after receiving the interaction synchronization confirmation response from all normal query initiation interfaces, extract the query intent corresponding to each normal query initiation interface, perform consistency verification on all query intents, and integrate the query intents that pass the verification into a unified query command.

4. The method according to claim 1, characterized in that, Determining the data source corresponding to each query domain information in the semantic parsing results includes: Based on the correspondence between domains and data sources in historical query records, construct a domain-data source correspondence table; Multiple query domain information is extracted from the semantic parsing results. Each query domain information is matched with the domain label in the corresponding table of the domain data source to obtain multiple candidate data sources corresponding to each query domain information. At least one candidate data source is extracted from the multiple candidate data sources corresponding to each query domain information to form multiple candidate data source groups. Calculate the overall domain coverage of each candidate data source group relative to all query domain information, and mark the candidate data source group with the largest overall domain coverage as the target data source group. If there are multiple target data source groups, retain the target data source group with the fewest candidate data sources. The candidate data sources in the target data source group are marked as the data sources corresponding to the query domain information in the semantic parsing results.

5. The method according to claim 1, characterized in that, Based on the preset retrieval rules of each data source, the unified query command is matched and retrieved to obtain initial query results. Then, based on these initial query results, multi-interface collaborative natural language data querying is performed, including: The search keywords are extracted from the unified query command, and the search keywords are converted into suitable search statements according to the preset search rules of each data source. The adapted search statement is sent to the corresponding target data source, and the adapted search statement and the data entries stored in the target data source are matched to obtain multiple matching data entries containing at least one search keyword. Calculate the relevance between the matched data entries and the unified query command, and arrange all the matched data entries according to the order of relevance to obtain the arrangement result; The first preset number of matching data entries in the sorting results are retained to form the initial query results; According to the output format requirements of each query initiation interface, the initial query results are converted to obtain text format output data, voice format output data and touch interaction format output data. The text format output data, the voice format output data, and the touch interaction format output data are respectively fed back to the corresponding query initiation interface to complete the natural language data query with multi-interface collaboration.

6. A multi-interface collaborative natural language data query system, characterized in that, include: The acquisition module is used to acquire text query data transmitted from the text input interface, voice query data transmitted from the voice input interface, and touch command data transmitted from the touch selection interface. An optimization module is used to optimize the voice query data and the touch command data respectively to obtain voice text data and touch text data, and to integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fused text set; The parsing module is used to perform semantic parsing on the multi-interface fused text set based on a preset natural language semantic parsing annotation dataset, and obtain semantic parsing results; The sending module is used to send the semantic parsing results to each query initiation interface through digital information transmission technology, and to process abnormal query initiation interfaces in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interactive synchronization confirmation requests, and then generate a unified query instruction. The determination module is used to determine the data source corresponding to each query domain information in the semantic parsing result; The retrieval module performs data matching and retrieval on the unified query command according to the preset retrieval rules of each data source to obtain initial query results, and then performs multi-interface collaborative natural language data query based on the initial query results; The voice query data and the touch command data are optimized separately to obtain voice text data and touch text data. The text query data, the voice text data, and the touch text data are then integrated to obtain a multi-interface fused text set, including: Acoustic features are extracted from the voice query data and combined with historical voice samples to construct voice matching relationships. The acoustic features include frequency features, amplitude features, and duration features. Based on the acoustic features and the speech matching relationship, the speech query data is divided into multiple text data segments. The pause intervals in the text data segments are semantically connected and spliced ​​together. The ambiguous pronunciations are corrected to obtain speech text data. A coordinate sequence is extracted from the touch command data, the coordinate sequence including a start coordinate point, an end coordinate point, and an intermediate coordinate point; Based on the preset functional area division table, the functional description corresponding to each coordinate point in the coordinate sequence is determined, each functional description is converted into a corresponding instruction text data segment, duplicate instruction data in the instruction text data segment is deduplicated, and valid instruction data in the conflicting instruction data in the instruction text data segment is filtered according to the preset priority rules to form touch text data. Based on the order in which the first timestamp of the text query data, the second timestamp of the voice text data, and the third timestamp of the touch text data are obtained, the text query data, the voice text data, and the touch text data are arranged and interface-marked to obtain a multi-interface fused text set. Based on a pre-defined natural language semantic parsing and annotation dataset, semantic parsing is performed on the multi-interface fused text set to obtain semantic parsing results, including: Multiple standard semantic items, multiple semantic relationship labels, and multiple domain classification labels are extracted from the pre-defined natural language semantic parsing and annotation dataset; Based on the semantic pause markers of natural language, the multi-interface fused text set is divided into multiple semantic segments, and the matching degree between each semantic segment and each standard semantic item is calculated. The standard semantic item corresponding to the highest numerical matching degree in each semantic segment is taken as the target semantic item of the corresponding semantic segment. The target semantic item, semantic relationship label and domain classification label of each semantic segment are associated to obtain multiple labeled semantic segments. Based on the original data order of the multi-interface fused text set, all tagged semantic fragments are arranged to obtain a semantic fragment set; Based on the semantic relationship tags of adjacent tagged semantic fragment pairs in the semantic fragment set, construct the association logic of each tagged semantic fragment in the semantic fragment set; The target semantic item, the semantic relationship label, the domain classification label, and the association logic are integrated to form a semantic parsing result.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of a multi-interface collaborative natural language data query method as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables a multi-interface collaborative natural language data query method as described in any one of claims 1 to 5.

Images