Multi-interface collaborative natural language data query method and system

Abstract

The invention provides a multi-interface collaborative natural language data query method and system, and relates to the technical field of natural language processing.The method comprises the steps that text query data transmitted by a text input interface, voice query data transmitted by a voice input interface and touch instruction data transmitted by a touch selection interface are obtained; performing optimization processing on the voice query data and the touch instruction data to obtain a multi-interface fusion text set; analyzing the annotation data set based on preset natural language semantics to obtain a semantic analysis result; generating a unified query instruction through a digital information transmission technology; determining a data source corresponding to each piece of query field information in the semantic analysis result; according to the method, the initial query result is obtained according to the preset retrieval rule of each data source, so that accurate analysis and cross-interface unified execution of the natural language query intention are realized, and the accuracy, robustness and interactive experience of data query in a complex scene are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a natural language data query method and system based on multi-interface cooperation. BACKGROUND

[0002] With the popularity of intelligent terminal devices and the diversification of human-computer interaction methods, the demand for natural language queries initiated by users in different scenarios through various ways such as text input, voice instruction or touch operation is increasing. In complex application environments such as smart office, intelligent vehicle and smart home, users often use multiple terminal devices such as mobile phones, tablets and computers at the same time, and expect the system to seamlessly integrate query requests from different input channels and realize cooperative response across devices and interfaces.

[0003] Currently, the existing mainstream scheme attempts to process query requests from different interfaces by building a unified intermediate semantic representation layer. The existing scheme usually converts the data of each input channel into preliminary text, and then independently analyzes it through a shared semantic understanding model, and directly forwards the analysis result to the corresponding data source to execute the query. However, the existing scheme has certain defects, for example, the data of each input channel lacks a cooperative optimization mechanism in the conversion process, which easily leads to deviations in the semantic representation generated by different channels, and the system has difficulty in determining whether multiple inputs are directed to the same query intent, thereby causing semantic understanding conflicts or inconsistent responses; the integration of multi-source inputs stays at the shallow splicing level and cannot deeply fuse context semantic information, resulting in a decrease in understanding accuracy in complex query scenarios, and difficulty in meeting the actual application requirements of high robustness and high cooperation, etc. SUMMARY

[0004] The present application aims to provide a natural language data query method and system based on multi-interface cooperation to solve the problems in the prior art that semantic understanding conflicts or inconsistent responses are easily caused, the understanding accuracy in complex query scenarios is insufficient, and it is difficult to meet the actual application requirements of high robustness and high cooperation, etc.

[0005] To solve the above technical problems, in a first aspect, the present application provides a natural language data query method based on multi-interface cooperation, comprising: acquiring text query data transmitted by a text input interface, voice query data transmitted by a voice input interface, and touch instruction data transmitted by a touch selection interface; optimizing the voice query data and the touch instruction data respectively to obtain voice text data and touch text data, and integrating the text query data, the voice text data and the touch text data to obtain a multi-interface fusion text set; perform semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result; The semantic analysis result is sent to each query initiation interface through a digital information transmission technology, and an abnormal query initiation interface in each query initiation interface is processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received an interaction synchronization confirmation request, and a unified query instruction is generated; Each query domain information in the semantic analysis result is determined to correspond to a data source; According to the preset retrieval rule of each data source, the unified query instruction is subjected to data matching retrieval to obtain an initial query result, so that the natural language data query of the multi-interface cooperation is performed according to the initial query result.

[0006] Optionally, the voice query data and the touch instruction data are respectively subjected to optimization processing to obtain voice text data and touch text data, and the text query data, the voice text data and the touch text data are integrated to obtain a multi-interface fusion text set, including: Acoustic features are extracted from the voice query data, and a voice matching relationship is constructed in combination with historical voice samples, the acoustic features including frequency features, amplitude features and time length features; Based on the acoustic features and the voice matching relationship, the voice query data is divided into a plurality of text data segments, a semantic coherent splicing process is performed on the pause interval part in the text data segment, and a voice correction process is performed on the fuzzy pronunciation part to obtain voice text data; Coordinate sequences are extracted from the touch instruction data, the coordinate sequences including starting coordinate points, ending coordinate points and intermediate coordinate points; Based on a preset function area division table, function descriptions corresponding to each coordinate point in the coordinate sequence are determined, each function description is converted into corresponding instruction text data segments, repeated instruction data in the instruction text data segments is subjected to a deduplication process, and effective instruction data in conflicting instruction data in the instruction text data segments is filtered according to a preset priority rule to form touch text data; According to the order of a first timestamp of the text query data, a second timestamp of the voice text data and a third timestamp of the touch text data, the text query data, the voice text data and the touch text data are subjected to arrangement and interface marking processing to obtain a multi-interface fusion text set.

[0007] Optionally, the multi-interface fusion text set is subjected to semantic analysis based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result, including: extract a plurality of standard semantic items, a plurality of semantic relationship labels and a plurality of domain classification labels from a preset natural language semantic parsing annotation data set; According to the semantic pause mark of the natural language, the multi-interface fusion text set is divided into a plurality of semantic segments, and the matching degree of each semantic segment with each standard semantic item is calculated; The standard semantic item corresponding to the maximum matching degree in each semantic segment is taken as the target semantic item of the corresponding semantic segment, and the target semantic item, the semantic relationship label and the domain classification label corresponding to each semantic segment are associated to obtain a plurality of labeled semantic segments; According to the original data sequence of the multi-interface fusion text set, all labeled semantic segments are arranged to obtain a semantic segment set; According to the semantic relationship labels of adjacent pairs of labeled semantic segments in the semantic segment set, the association logic of each labeled semantic segment in the semantic segment set is constructed; The target semantic item, the semantic relationship label, the domain classification label and the association logic are integrated to form a semantic parsing result.

[0008] Optionally, according to the semantic relationship labels of adjacent pairs of labeled semantic segments in the semantic segment set, the association logic of each labeled semantic segment in the semantic segment set is constructed, comprising: The semantic relationship labels of all adjacent pairs of labeled semantic segments in the semantic segment set are parsed to obtain the association type of each adjacent pair of labeled semantic segments, and the association type includes a cause-effect relationship, a parallel relationship and a progressive relationship; The matching degree of each association type and a preset relationship type rule is calculated, and the adjacent pair of labeled semantic segments corresponding to the association type with a matching degree exceeding a preset matching threshold is added with an association type identifier to form a segment relationship chain; Based on a preset association type hierarchy rule, the association types in the segment relationship chain are hierarchically divided to obtain association types of different priorities; The association types of different priorities are compared with a preset priority to obtain a priority comparison result, so that according to the priority comparison result, each adjacent pair of labeled semantic segments in the segment relationship chain is rearranged to obtain an adjusted segment relationship chain; According to the adjusted segment relationship chain, the association logic of the semantic segment set is constructed.

[0009] Optionally, the semantic parsing result is sent to each query initiation interface through digital information transmission technology, and abnormal query initiation interfaces in each query initiation interface are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have received an interaction synchronization confirmation request, and a unified query instruction is generated, comprising: data-convert the semantic analysis result based on a transmission protocol of each query initiation interface to obtain interface-compatible data, the interface-compatible data including first compatible data corresponding to a character input interface, second compatible data corresponding to a voice input interface, and third compatible data corresponding to a touch selection interface; send the interface-compatible data to the corresponding query initiation interface through a digital information transmission technology, and record receiving state information of each query initiation interface, the receiving state information including a receiving feedback time and a receiving integrity identifier; identify an abnormal query initiation interface with abnormal receiving state information according to the receiving state information, the abnormal query initiation interface including a query initiation interface with a receiving feedback time exceeding a preset time threshold or a query initiation interface with an incomplete receiving integrity identifier; send corresponding interface-compatible data to all abnormal query initiation interfaces to update the receiving state information of all abnormal query initiation interfaces until all abnormal query initiation interfaces are updated to normal query initiation interfaces; send an interaction synchronization confirmation request to all normal query initiation interfaces, and after receiving interaction synchronization confirmation responses from all normal query initiation interfaces, extract query intents corresponding to each normal query initiation interface, and perform consistency verification on all query intents, and integrate query intents passing the verification into a unified query instruction.

[0010] Optionally, determining data sources corresponding to each query domain information in the semantic analysis result includes: constructing a domain-data source correspondence table based on a correspondence relationship between domains and data sources in historical query records; extracting a plurality of query domain information from the semantic analysis result, matching each query domain information with a domain tag in the domain-data source correspondence table to obtain a plurality of candidate data sources corresponding to each query domain information, and extracting at least one candidate data source from the plurality of candidate data sources corresponding to each query domain information to form a plurality of candidate data source groups; calculating an overall domain coverage of each candidate data source group with respect to all query domain information, and marking a candidate data source group with the largest overall domain coverage as a target data source group, and if there are multiple target data source groups, retaining the target data source group with the least number of candidate data sources; marking the candidate data sources in the target data source group as data sources corresponding to each query domain information in the semantic analysis result.

[0011] Optionally, performing data matching retrieval on the unified query instruction according to a preset retrieval rule of each data source to obtain an initial query result, and performing multi-interface collaborative natural language data query based on the initial query result includes: extracting a search keyword from the unified query instruction, and converting the search keyword into an adaptive search statement according to a preset search rule of each data source; sending the adaptive search statement to a corresponding target data source, and matching the adaptive search statement with data entries stored by the target data source to obtain a plurality of matching data entries containing at least one search keyword; calculating a relevance degree of the matching data entries with respect to the unified query instruction, and arranging all the matching data entries according to a high-low order of the relevance degrees to obtain an arrangement result; retaining a front preset number of matching data entries in the arrangement result to form an initial query result; format-converting the initial query result according to output format requirements of each query initiation interface to obtain text format output data, voice format output data and touch interaction format output data; feeding the text format output data, the voice format output data and the touch interaction format output data to corresponding query initiation interfaces respectively to complete the multi-interface cooperative natural language data query.

[0012] In a second aspect, the present application provides a multi-interface cooperative natural language data query system, comprising: an acquisition module configured to acquire text query data transmitted by a text input interface, voice query data transmitted by a voice input interface and touch instruction data transmitted by a touch selection interface; an optimization module configured to perform optimization processing on the voice query data and the touch instruction data respectively to obtain voice text data and touch text data, and to integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fusion text set; an analysis module configured to perform semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result; a sending module configured to send the semantic analysis result to each query initiation interface through digital information transmission technology, and to process an abnormal query initiation interface in each query initiation interface until it is confirmed that all the query initiation interfaces are normal query initiation interfaces and have all received an interaction synchronization confirmation request, and then generate a unified query instruction; a determination module configured to determine data sources corresponding to each query field information in the semantic analysis result; a search module configured to perform data matching search on the unified query instruction according to a preset search rule of each data source to obtain an initial query result, and to perform multi-interface cooperative natural language data query according to the initial query result.

[0013] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the method for multi-interface coordinated natural language data query according to the first aspect.

[0014] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable by a processor to implement the steps of the method for multi-interface coordinated natural language data query according to the first aspect.

[0015] The method comprises the following steps: acquiring text query data transmitted by a character input interface, voice query data transmitted by a voice input interface, and touch instruction data transmitted by a touch selection interface; respectively optimizing the voice query data and the touch instruction data to obtain voice text data and touch text data; integrating the text query data, the voice text data, and the touch text data to obtain a multi-interface fusion text set; performing semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result; sending the semantic analysis result to each query initiation interface through digital information transmission technology, and processing an abnormal query initiation interface in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received an interaction synchronization confirmation request, and generating a unified query instruction; determining data sources corresponding to each query field information in the semantic analysis result; performing data matching retrieval on the unified query instruction according to a preset retrieval rule of each data source to obtain an initial query result, so as to perform multi-interface cooperative natural language data query according to the initial query result; and comprehensively collecting multi-modal input sources by acquiring the text query data transmitted by the character input interface, the voice query data transmitted by the voice input interface, and the touch instruction data transmitted by the touch selection interface, ensuring that the system can simultaneously respond to query requests initiated by a user through different interaction modes, and solving the understanding deviation problem caused by missing or errors of a single channel information; the structured analysis of the fused text by using a standardized semantic item and a label system improves the understanding accuracy and the field recognition ability of the system for complex multi-source input semantic intent; the response consistency of each channel before the query instruction is generated and the robustness of the system as a whole are ensured; the precise directional retrieval in a multi-source heterogeneous data environment is realized, the relevance and integrity of the query result are ensured, and the whole process of the multi-interface cooperative natural language data query is completed; further, the method comprises the following steps: extracting a standard semantic item, a semantic relationship label, and a field classification label from the preset natural language semantic analysis annotation data set, and selecting the standard semantic item with the highest matching degree as a target semantic item of each semantic segment; associating the target semantic item with a corresponding semantic relationship label and a field classification label to form a labeled semantic segment; constructing an association logic between adjacent segments according to the semantic relationship label, integrating the target semantic item, the semantic relationship label, the field classification label, and the association logic, and generating a complete semantic analysis result; solving the inconsistent understanding problem caused by semantic expression deviation of each input channel, enhancing the depth understanding ability of the system for context semantics and the intent recognition accuracy in a complex query scene, and thus improving the overall semantic consistency and response reliability of the multi-interface cooperative query. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative labor based on these drawings also belong to the protection scope of the present application.

[0017] Figure 1 A flowchart of a multi-interface cooperative natural language data query method provided by an embodiment of the present application; Figure 2 A specific implementation schematic diagram of a multi-interface cooperative natural language data query method provided by an embodiment of the present application; Figure 3 A structural schematic diagram of a multi-interface cooperative natural language data query system provided by an embodiment of the present application; DETAILED DESCRIPTION

[0018] In view of the demand of users for concurrent natural language query through multiple ways such as text, voice, touch, etc. in the multi-terminal multi-interface cooperative scene of smart office, intelligent vehicle and smart home, etc., in order to solve the problems of semantic representation deviation caused by independent conversion and analysis of data of each input channel, shallow integration of multi-source information, inconsistent cross-interface response, etc. in the prior art, the present application realizes comprehensive coverage of multi-modal input by uniformly collecting text query data of a text input interface, voice query data of a voice input interface and touch instruction data of a touch selection interface; the semantic expression quality of non-text input is improved by performing targeted optimization processing on voice and touch data, and the three types of data are fused into a unified multi-interface fusion text set, breaking through the limitation of independent semantic analysis of each channel in the prior art, and enhancing the integrity and consistency of input information; further, the text set after fusion is subjected to deep semantic analysis by using a preset natural language semantic analysis annotation data set, the analysis result is fed back to each query initiation interface through digital information transmission technology, ensuring that all interfaces reach interactive consistency before generating a unified query instruction, and avoiding incorrect instruction sending caused by interface abnormality or response delay; the corresponding data source is determined based on the analysis result and directional retrieval is performed, realizing efficient cooperative query of multi-source heterogeneous data, so as to build a natural language data query system with more accurate semantic understanding, more reliable interface cooperation and more smooth user experience.

[0019] In order to make the person skilled in the art better understand the present application, the present application will be further described in detail below in combination with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor also belong to the protection scope of the present application.

[0020] The core of the present application is to provide a multi-interface cooperative natural language data query method, and a flowchart of a specific embodiment thereof is shown in Figure 1 The method comprises the following steps. Step 101: obtaining text query data transmitted by a text input interface, voice query data transmitted by a voice input interface, and touch instruction data transmitted by a touch selection interface.

[0021] In this step, the text input interface refers to an interface for receiving a user's input text form query request; the text query data refers to user text form query data transmitted by the text input interface; the voice input interface refers to an interface for receiving a user's input voice form query request; the voice query data refers to user voice form query data transmitted by the voice input interface; the touch selection interface refers to an interface for receiving a user's touch operation form query request; and the touch instruction data refers to user touch operation form query instruction data transmitted by the touch selection interface.

[0022] In the embodiment of the present application, the text input interface, the voice input interface and the touch selection interface are started to respectively receive information transmitted by a user through the three interfaces, that is, text query data input by the user is obtained through the text input interface, voice query data input by the user is obtained through the voice input interface, and touch instruction data generated by touch operation of the user is obtained through the touch selection interface, thereby completing initial acquisition of multi-interface query data.

[0023] Step 102: respectively optimizing the voice query data and the touch instruction data to obtain voice text data and touch text data, integrating the text query data, the voice text data and the touch text data to obtain a multi-interface fusion text set.

[0024] In this step, the voice text data refers to text form data obtained by optimizing the voice query data; the touch text data refers to text form data obtained by optimizing the touch instruction data; and the multi-interface fusion text set refers to a unified text set formed by integrating the text query data, the voice text data and the touch text data.

[0025] Step 103: based on a preset natural language semantic analysis annotation data set, performing semantic analysis on the multi-interface fusion text set to obtain a semantic analysis result.

[0026] In this step, the preset natural language semantic analysis annotation data set refers to an annotation data set pre-constructed to contain standard semantic items, semantic relationship labels and domain classification labels; and the semantic analysis result refers to a result set obtained by performing semantic analysis on the multi-interface fusion text set.

[0027] Step 104: sending the semantic analysis result to each query initiation interface through a digital information transmission technology, processing an abnormal query initiation interface in each query initiation interface, and generating a unified query instruction after confirming that all query initiation interfaces are normal query initiation interfaces and have received an interactive synchronization confirmation request.

[0028] In this step, the digital information transmission technology refers to a technology for realizing data transmission between different interfaces; the query initiation interface refers to an interface initiating a query request; the abnormal query initiation interface refers to a query initiation interface receiving an abnormal semantic analysis result; the normal query initiation interface refers to a query initiation interface receiving a normal semantic analysis result; the interactive synchronization confirmation request refers to a request for confirming whether each query initiation interface completes semantic analysis result reception and interactive synchronization; and the unified query instruction refers to a unified query instruction formed after integrating the query intentions of each query initiation interface passing the verification.

[0029] Step 105: determining a data source corresponding to each query field information in the semantic analysis result.

[0030] In this step, the query field information refers to knowledge field category information of a query request extracted from the semantic analysis result; and the data source refers to a source for providing data required by a query.

[0031] Step 106: performing data matching retrieval on the unified query instruction according to a preset retrieval rule of each data source to obtain an initial query result, and performing multi-interface collaborative natural language data query according to the initial query result.

[0032] In this step, the preset retrieval rule refers to a rule preset by each data source for data retrieval; the data matching retrieval refers to a process of finding matching data of the unified query instruction in the data source according to the preset retrieval rule; and the initial query result refers to a preliminary query result obtained by data matching retrieval.

[0033] The embodiment of the application realizes multi-interface collaborative natural language data query, improves the utilization rate of query data, the accuracy of semantic analysis, the reliability of interface interaction, and the precision of data retrieval, and meets the query requirements of users in multiple scenarios.

[0034] The application provides one specific embodiment, step 102, respectively optimizing the voice query data and the touch instruction data to obtain voice text data and touch text data, integrating the text query data, the voice text data, and the touch text data to obtain a multi-interface fusion text set, specifically including the following steps. Step 201: extracting acoustic features from the voice query data, constructing a voice matching relationship in combination with historical voice samples, the acoustic features including frequency features, amplitude features and time length features.

[0035] In this step, the acoustic features refer to features extracted from the voice query data for representing voice characteristics; the historical voice samples refer to stored voice data for reference in the past; the voice matching relationship refers to a corresponding relationship between the acoustic features and standard voice units constructed based on the historical voice samples; the frequency features refer to features representing frequency properties of voice signals; the amplitude features refer to features representing vibration amplitude properties of voice signals; and the time length features refer to features representing time length properties of voice signals.

[0036] In the embodiments of the present application, the acquired voice query data is subjected to signal analysis, and frequency features, amplitude features and time length features capable of reflecting voice characteristics are extracted therefrom, which together constitute acoustic features; stored historical voice samples are called, the samples containing a plurality of standard voice units and corresponding acoustic feature data, and a voice matching relationship between the current acoustic features and the standard voice units is established by comparing the acoustic features of the current voice query data with the acoustic features of the standard voice units in the historical voice samples, thereby laying a foundation for subsequent conversion of voice query data into text data.

[0037] Step 202: dividing the voice query data into a plurality of text data segments based on the acoustic features and the voice matching relationship, performing semantic coherent splicing processing on pause interval portions in the text data segments, and performing voice correction processing on ambiguous pronunciation portions, to obtain voice text data.

[0038] In this step, the text data segment refers to a fragmented text obtained by dividing the voice query data; the pause interval portion in the text data segment refers to an incoherent portion in the text data segment due to voice pause; and the ambiguous pronunciation portion refers to an ambiguous portion in the text data segment due to unclear pronunciation.

[0039] In the embodiments of the present application, the voice query data is divided according to the boundaries of voice units based on the constructed acoustic features and voice matching relationship, and is converted into a plurality of independent text data segments; for each text data segment, a pause interval portion in the text data segment due to voice pause is identified, context semantic logic is combined to supplement the coherence content so that the portion forms a coherent expression; meanwhile, an ambiguous pronunciation portion in the text data segment due to ambiguous pronunciation is identified, and the ambiguous portion is corrected by comparing the text corresponding to the standard pronunciation in the same or similar context in the historical voice samples, to ensure accurate text expression, and finally complete and accurate voice text data is obtained.

[0040] Step 203: extracting a coordinate sequence from the touch instruction data, the coordinate sequence including a starting coordinate point, an ending coordinate point, and intermediate coordinate points.

[0041] In this step, the coordinate sequence refers to a set of coordinates extracted from the touch instruction data and representing a touch operation trajectory; the starting coordinate point refers to a coordinate corresponding to a starting position of the touch operation in the coordinate sequence; the ending coordinate point refers to a coordinate corresponding to an ending position of the touch operation in the coordinate sequence; and the intermediate coordinate points refer to coordinates corresponding to touch positions between the starting coordinate point and the ending coordinate point in the coordinate sequence.

[0042] In the embodiments of the present application, the touch instruction data is analyzed to extract a coordinate sequence recording a user touch operation path, which includes a starting coordinate point when the user starts touching, an ending coordinate point when the user stops touching, and all intermediate coordinate points during the touch process, thereby completely presenting a touch operation trajectory of the user and providing data support for determining a function corresponding to the touch operation.

[0043] Step 204: determining a function description corresponding to each coordinate point in the coordinate sequence based on a preset function area division table, converting each function description into a corresponding instruction text data segment, performing deduplication processing on repeated instruction data in the instruction text data segment, and filtering valid instruction data in conflicting instruction data in the instruction text data segment according to a preset priority rule to form touch text data.

[0044] In this step, the preset function area division table refers to a table prepared in advance for specifying functions corresponding to each area of a touch interface; the function description refers to a function explanation of a touch operation determined according to an area where a coordinate point is located; the instruction text data segment refers to an instruction segment in the form of text obtained by converting a function description; the repeated instruction data refers to instruction data with the same content in the instruction text data segment; the preset priority rule refers to a priority standard preset for filtering conflicting instruction data; the conflicting instruction data refers to instruction data with mutually contradictory content in the instruction text data segment; and the valid instruction data refers to instruction data filtered from the conflicting instruction data and conforming to the preset priority rule.

[0045] In the embodiments of the present application, the preset function area division table is called, and a function description corresponding to each coordinate point is determined by searching the table according to a coordinate range where each coordinate point is located; each function description is converted into an instruction text data segment capable of embodying an instruction meaning according to a preset text conversion rule; all instruction text data segments are compared in content to identify repeated instruction data with the same content, and the repeated data is deleted through deduplication processing; if there are conflicting instruction data with mutually contradictory content, valid instruction data conforming to a preset priority rule is filtered according to the priority rule, and the valid instruction data is integrated to form touch text data.

[0046] Step 205: According to the order of the first timestamp of the text query data, the second timestamp of the voice text data, and the third timestamp of the touch text data, the text query data, the voice text data, and the touch text data are arranged and interface marked to obtain a multi-interface fusion text set.

[0047] In this step, the first timestamp refers to an identifier recording the time of obtaining the text query data; the second timestamp refers to an identifier recording the time of obtaining the voice text data; and the third timestamp refers to an identifier recording the time of obtaining the touch text data.

[0048] In the embodiments of the present application, the time information recorded during data acquisition is called, wherein the first timestamp corresponds to the acquisition time of the text query data, the second timestamp corresponds to the acquisition time of the voice text data, and the third timestamp corresponds to the acquisition time of the touch text data; the text query data, the voice text data, and the touch text data are sorted according to the order of the three timestamps to ensure that the data is presented in the time order of user operation; and the corresponding interface source mark is added to each type of data to clearly indicate the source of the data, and finally a multi-interface fusion text set is formed.

[0049] The embodiments of the present application solve the problem of heterogeneous data format, incomplete or inaccurate expression of multi-interface query data by extracting acoustic features from voice query data and constructing a voice matching relationship with historical voice samples to process the voice query data into accurate voice text data, extracting coordinate sequences from touch instruction data and processing them into accurate touch text data according to a pre-set function area division table, and then sorting and interface marking the three types of data according to the timestamps to form a multi-interface fusion text set, which provides a high-quality and unified data basis for subsequent semantic analysis, and improves the efficiency and accuracy of multi-interface collaborative query.

[0050] The present application provides a specific embodiment as shown in Figure 2 Step 103: Based on the pre-set natural language semantic analysis annotation data set, the multi-interface fusion text set is subjected to semantic analysis to obtain a semantic analysis result, which specifically includes the following steps: Step 301: Extract a plurality of standard semantic items, a plurality of semantic relationship labels, and a plurality of domain classification labels from the pre-set natural language semantic analysis annotation data set.

[0051] In this step, the standard semantic item refers to a standard semantic unit for reference in the pre-set natural language semantic analysis annotation data set; the semantic relationship label refers to a label for identifying the relationship between semantics in the pre-set natural language semantic analysis annotation data set; and the domain classification label refers to a label for identifying the knowledge domain to which the semantics belong in the pre-set natural language semantic analysis annotation data set.

[0052] In the embodiment of the present application, a preset natural language semantic parsing annotation dataset is called, a plurality of standard semantic items used for comparison with subsequent semantic segments are screened out from the dataset, these standard semantic items cover core semantics in common query scenarios; meanwhile, a plurality of semantic relationship labels used for describing the association mode between different semantics and a plurality of domain classification labels used for dividing the knowledge domain to which the semantics belong are extracted, thereby providing a reference basis for subsequent semantic segment parsing.

[0053] Step 302: According to the semantic pause mark of the natural language, the plurality of interface fusion text sets are divided into a plurality of semantic segments, and the matching degree of each semantic segment with each standard semantic item is calculated.

[0054] In this step, the semantic pause mark of the natural language refers to the pause mark used to distinguish different semantic units in the natural language, such as comma, period, etc.; the semantic segment refers to an independent semantic unit obtained by dividing the plurality of interface fusion text sets according to the semantic pause mark; and the matching degree refers to a quantitative value of the similarity between the semantic segment and the standard semantic item.

[0055] In the embodiment of the present application, the plurality of interface fusion text sets are subjected to text analysis, the semantic pause mark of the natural language is recognized, and the plurality of interface fusion text sets are split into a plurality of semantic segments capable of independently expressing complete semantics according to these marks as the division boundary; for each semantic segment, the keywords contained therein are compared with the keywords contained in each standard semantic item, and the matching degree = (the number of overlapping keywords between the semantic segment and the standard semantic item / the total number of keywords of the standard semantic item) x 100% is calculated, thereby obtaining the matching degree value of each semantic segment with each standard semantic item.

[0056] Step 303: The standard semantic item corresponding to the maximum matching degree value in each semantic segment is taken as the target semantic item of the corresponding semantic segment, and the target semantic item corresponding to each semantic segment, the semantic relationship label and the domain classification label are associated, thereby obtaining a plurality of labeled semantic segments.

[0057] In this step, the target semantic item refers to the standard semantic item with the maximum matching degree value in the semantic segment; and the labeled semantic segment refers to the semantic segment associated with the target semantic item, the semantic relationship label and the domain classification label.

[0058] In the embodiment of the present application, the plurality of matching degree values corresponding to each semantic segment are compared, the maximum matching degree is screened out, the standard semantic item corresponding to the maximum matching degree is determined as the target semantic item of the semantic segment, the association information of the target semantic item in the preset natural language semantic analysis annotation data set is found, the semantic relation label and the domain classification label corresponding to the target semantic item are found, the target semantic item, the corresponding semantic relation label and the domain classification label are bound with the original semantic segment, so that each semantic segment has a clear semantic identification, and a plurality of labeled semantic segments are formed.

[0059] Step 304: arranging all the labeled semantic segments according to the original data order of the multi-interface fusion text set to obtain a semantic segment set.

[0060] In this step, the original data order refers to the original arrangement order of each data in the multi-interface fusion text set; the semantic segment set refers to the set formed after arranging all the labeled semantic segments according to the original data order.

[0061] In the embodiment of the present application, the original data order recorded when the multi-interface fusion text set is integrated is called, which reflects the logical order of user operations and the original association of data; all the labeled semantic segments obtained before are arranged in sequence according to the original data order, so that the order of the labeled semantic segments is consistent with the logical order of user operations and the original association of data, and the semantic logic is prevented from being broken due to disorder, and the semantic segment set is formed after the arrangement is completed.

[0062] Step 305: constructing the association logic of each labeled semantic segment in the semantic segment set according to the semantic relation labels of adjacent labeled semantic segment pairs in the semantic segment set.

[0063] In this step, the adjacent labeled semantic segment pair refers to a pair of two labeled semantic segments adjacent in position in the semantic segment set; the association logic refers to the logical relationship used to describe the semantic association between each labeled semantic segment in the semantic segment set.

[0064] In the embodiment of the present application, the semantic segment set is traversed, all the adjacent labeled semantic segments are identified, each adjacent labeled semantic segment pair is composed of two adjacent labeled semantic segments; the semantic relation labels of the two labeled semantic segments in each adjacent labeled semantic segment pair are extracted, the semantic association mode between the adjacent labeled semantic segment pairs is determined by analyzing the meanings and mutual association of the two semantic relation labels; based on the semantic association modes of all the adjacent labeled semantic segment pairs, the overall semantic connection between all the labeled semantic segments in the semantic segment set is sorted out, and the association logic reflecting the logical relationship of each labeled semantic segment is constructed.

[0065] Step 306: integrating the target semantic item, the semantic relationship label, the domain classification label and the association logic to form a semantic analysis result.

[0066] In the embodiment of the present application, the target semantic item, the semantic relationship label and the domain classification label corresponding to all the labeled semantic fragments are collected, and these information is integrated with the constructed association logic; it is ensured that each target semantic item can be associated with other target semantic items through the semantic relationship label, and a semantic analysis result containing complete semantic information and logical association is formed.

[0067] The embodiment of the present application solves the problems of semantic fragmentation and unclear association of the multi-interface fusion text set, provides clear and structured semantic basis for subsequent generation of unified query instruction, and improves the semantic understanding accuracy of multi-interface collaborative query.

[0068] For example, the preset natural language semantic analysis annotation data set contains standard semantic items such as query device parameters and query device installation guide, semantic relationship labels such as parallel relationship, progressive relationship and independent relationship, and domain classification labels such as device information query domain, and these standard semantic items, semantic relationship labels and domain classification labels are extracted from the data set. The multi-interface fusion text set formed before is [text input interface] B type device parameters; [voice input interface] query installation guide of B type device; [touch selection interface] query installation guide of B type device, according to the semicolons therein, it is divided into three semantic fragments of [text input interface] B type device parameters, [voice input interface] query installation guide of B type device and [touch selection interface] query installation guide of B type device.

[0069] The matching degree of each semantic fragment with each standard semantic item is calculated = (the number of overlapping keywords of the semantic fragment and the standard semantic item / the total number of keywords of the standard semantic item) x 100%. For the first semantic fragment [text input interface] B type device parameters, the keywords extracted therefrom are B type device and parameter; the keywords of the query device parameter standard semantic item are device and parameter, the number of overlapping keywords is 2, the total number of keywords is 2, and the matching degree is calculated = (2 / 2) x 100% = 100%; the keywords of the query device installation guide standard semantic item are device and installation guide, the number of overlapping keywords is 1, and the total number of keywords is 2, and the matching degree is calculated = (1 / 2) x 100% = 50%.

[0070] For the second semantic fragment [voice input interface] to query the installation guide of the B model device, the extracted keywords are query, B model device, and installation guide; the keywords for querying the standard semantic item of device installation guide are query, device, and installation guide, the number of coinciding keywords is 3, the total number of keywords is 3, and the matching degree is calculated as = (3 / 3) * 100% = 100%; the keywords for querying the standard semantic item of device parameter are query, device, and parameter, the number of coinciding keywords is 2, the total number of keywords is 3, and the matching degree is calculated as = (2 / 3) * 100% ≈ 66.7%.

[0071] For the third semantic fragment [touch selection interface] to query the installation guide of the B model device, the extracted keywords are query, B model device, and installation guide; the keywords for querying the standard semantic item of device installation guide are query, device, and installation guide, which coincide with the keywords of the third semantic fragment, the number of coinciding keywords is 3, the total number of keywords for querying the standard semantic item of device installation guide is 3, and the matching degree is calculated as = (3 / 3) * 100% = 100%; the keywords for querying the standard semantic item of device parameter are query, device, and parameter, which coincide with the keywords of the third semantic fragment, the number of coinciding keywords is 2, the total number of keywords for querying the standard semantic item of device parameter is 3, and the matching degree is calculated as = (2 / 3) * 100% ≈ 66.7%.

[0072] The target semantic item of the first semantic fragment is determined as querying device parameter, and the device information query domain label and the independent relationship label are associated according to the preset data set; the target semantic items of the second and third semantic fragments are determined as querying device installation guide, and the device information query domain label and the parallel relationship label are associated, forming three labeled semantic fragments. The three labeled semantic fragments are arranged into a semantic fragment set in the order of the original data of the multi-interface fusion text set. The adjacent labeled semantic fragment pairs in the semantic fragment set are identified, the semantic relationship labels of the first and second fragment pairs are independent relationship and parallel relationship respectively, and it is concluded that the association type is parallel relationship by analyzing that the semantics of both revolves around querying the B model device; the semantic relationship labels of the second and third fragment pairs are both parallel relationship, and it is concluded that the association type is parallel relationship by analyzing. Based on these associations, an association logic is constructed that the three fragments all belong to the device information query domain and are in parallel relationship with each other, and the target semantic item, the semantic relationship label, the domain classification label, and the association logic are integrated to form a semantic analysis result.

[0073] The present application provides a specific embodiment, step 305, constructing the association logic of each labeled semantic fragment in the semantic fragment set according to the semantic relationship labels of the adjacent labeled semantic fragment pairs in the semantic fragment set, specifically including the following steps: Step 311: parsing the semantic relation labels of all adjacent labeled semantic segment pairs in the semantic segment set to obtain the association types of the adjacent labeled semantic segment pairs, the association types including a cause-effect relation, a parallel relation and a progressive relation.

[0074] In this step, the association type refers to a category used to describe the semantic relation between adjacent labeled semantic segment pairs; the cause-effect relation refers to a relation in the association type indicating that one semantic segment is the cause or result of another semantic segment; the parallel relation refers to a relation in the association type indicating that the contents of multiple semantic segments are parallel and have no precedence relation; and the progressive relation refers to a relation in the association type indicating that the contents of the semantic segments are deepened layer by layer.

[0075] In the embodiments of the present application, all positionally adjacent labeled semantic segments are filtered out from the semantic segment set to form multiple adjacent labeled semantic segment pairs; the semantic relation labels of the two labeled semantic segments in each adjacent labeled semantic segment pair are extracted, and the semantic logic expressed by the two semantic relation labels is analyzed to determine the association type corresponding to each adjacent labeled semantic segment pair from the cause-effect relation, the parallel relation and the progressive relation, thereby obtaining the association types of the adjacent labeled semantic segment pairs.

[0076] Step 312: calculating the matching degrees of the association types with respect to the preset relation type rule, and adding an association type identifier to the adjacent labeled semantic segment pairs corresponding to the association types whose matching degrees exceed a preset matching threshold to form a segment relation chain.

[0077] In this step, the preset relation type rule refers to a rule prepared in advance for judging the validity of the association type; the preset matching threshold refers to a matching degree threshold value set in advance for screening valid association types; the association type identifier refers to an identifier used to mark the association type of the adjacent labeled semantic segment pair; and the segment relation chain refers to a chain formed by sequentially connecting the adjacent labeled semantic segment pairs after adding the association type identifier.

[0078] In the embodiments of the present application, the preset relation type rule is called, which specifies the determination criteria corresponding to each association type; for each association type, the features thereof are compared with the determination criteria of the preset relation type rule to calculate the matching degree = (the number of coinciding items of the association type features and the determination criteria of the preset rule / the total number of the determination criteria of the preset rule) x 100%; the calculated matching degree is compared with the preset matching threshold, and the association types whose matching degrees exceed the preset matching threshold are retained, an association type identifier capable of reflecting the association types of the adjacent labeled semantic segment pairs corresponding to the association types is added, and then the adjacent labeled semantic segment pairs with the identifier are sequentially connected in the original order of the semantic segment set to form a segment relation chain.

[0079] Step 313: Hierarchically dividing the association types in the segment relationship chain based on a preset association type hierarchy rule to obtain association types with different priorities.

[0080] In this step, the preset association type hierarchy rule refers to a rule preset for dividing the priority order of association types; and the association types with different priorities refer to association types with different priority levels after being divided according to the preset association type hierarchy rule.

[0081] In the embodiments of the present application, the preset association type hierarchy rule is called, which explicitly defines the priority order of association types; according to the rule, all association types in the segment relationship chain are hierarchically divided, the association types meeting the cause-effect relationship rule are divided into the highest priority, the association types meeting the progressive relationship rule are divided into the intermediate priority, and the association types meeting the parallel relationship rule are divided into the lowest priority, to obtain association types with different priorities and explicitly define the priority order of each association type in the segment relationship chain.

[0082] Step 314: Comparing the association types with different priorities with a preset priority to obtain a priority comparison result, and rearranging each adjacent labeled semantic segment pair in the segment relationship chain according to the priority comparison result to obtain an adjusted segment relationship chain.

[0083] In this step, the preset priority refers to a preset association type priority standard for comparison; the priority comparison result refers to a result obtained after comparing the association types with different priorities with the preset priority; and the adjusted segment relationship chain refers to a chain formed after rearranging the original segment relationship chain according to the priority comparison result.

[0084] In the embodiments of the present application, the preset priority is called, and the actual priority of each association type in the segment relationship chain is compared with the preset priority one by one to determine whether the actual priority is consistent with the preset priority, and if so, it is marked as consistent, and if not, it is marked as inconsistent, to obtain a priority comparison result; for the adjacent labeled semantic segment pair corresponding to the association type that does not meet the priority comparison result, the position of the pair in the segment relationship chain is adjusted according to the preset priority, and the adjusted segment relationship chain is obtained after rearrangement.

[0085] Step 315: Constructing the association logic of the semantic segment set according to the adjusted segment relationship chain.

[0086] In the embodiment of the present application, the association type identifiers and the arrangement orders of each adjacent pair of tagged semantic segments in the adjusted segment relationship chain are analyzed, and the semantic association paths among all the tagged semantic segments are sorted out; the logical order and the association mode of each tagged semantic segment are determined according to the association paths, and the logical relations are integrated to form an association logic capable of fully reflecting the semantic relations among all the tagged semantic segments in the semantic segment set.

[0087] The embodiment of the present application solves the problems of mixed association types and unclear priorities of multi-interface semantic segments, ensures the accuracy and order of the semantic association logic, and provides clear logical support for efficient application of subsequent semantic analysis results.

[0088] The present application provides an embodiment, step 104, the semantic analysis result is sent to each query initiation interface through digital information transmission technology, and abnormal query initiation interfaces in each query initiation interface are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have received the interactive synchronization confirmation request, and a unified query instruction is generated, which specifically includes the following steps: Step 401: Based on the transmission protocol of each query initiation interface, the semantic analysis result is data-converted to obtain interface-compatible data, the interface-compatible data including first compatible data corresponding to the text input interface, second compatible data corresponding to the voice input interface, and third compatible data corresponding to the touch selection interface.

[0089] In this step, the transmission protocol refers to the rule standard used by each query initiation interface for data transmission; the data conversion refers to the operation of converting the semantic analysis result into a format conforming to the transmission protocol of each query initiation interface; the interface-compatible data refers to data that can be recognized and received by each query initiation interface after data conversion; the first compatible data refers to interface-compatible data matching the transmission protocol of the text input interface; the second compatible data refers to interface-compatible data matching the transmission protocol of the voice input interface; and the third compatible data refers to interface-compatible data matching the transmission protocol of the touch selection interface.

[0090] In the embodiment of the present application, the transmission protocol corresponding to each query initiation interface is first determined, and the transmission protocol formats of different interfaces are different; according to the transmission protocol requirements of each interface, the semantic analysis result is data-converted, and the target semantic items, semantic relation labels and other information in the semantic analysis result are converted into first compatible data recognizable by the text input interface, second compatible data recognizable by the voice input interface, and third compatible data recognizable by the touch selection interface, to finally obtain interface-compatible data containing the first compatible data, the second compatible data and the third compatible data.

[0091] Step 402: sending the interface compatible data to the corresponding query initiating interface through digital information transmission technology, and recording the receiving state information of each query initiating interface, the receiving state information including receiving feedback time and receiving integrity identification.

[0092] In this step, the receiving state information refers to the information recording the receiving state of the interface compatible data by each query initiating interface; the receiving feedback time refers to the time of the feedback of the receiving result after the query initiating interface receives the interface compatible data; and the receiving integrity identification refers to the identification for identifying whether the interface compatible data received by the query initiating interface is complete.

[0093] In the embodiment of the present application, the first compatible data is sent to the character input interface, the second compatible data is sent to the voice input interface, and the third compatible data is sent to the touch selection interface through digital information transmission technology; at the same time of sending the data, the timing function is started to record the time from the receiving of the interface compatible data to the feedback of the received signal by each query initiating interface, which is the receiving feedback time; at the same time, the data integrity checking instruction is sent to each query initiating interface, and each interface performs the integrity checking on the received compatible data according to the checking instruction to generate the receiving integrity identification of completeness or incompleteness; the receiving feedback time and the receiving integrity identification of each query initiating interface are integrated to form the receiving state information of each query initiating interface.

[0094] Step 403: identifying the abnormal query initiating interface with abnormal receiving state information according to the receiving state information, the abnormal query initiating interface including the query initiating interface with the receiving feedback time exceeding the preset time threshold or the query initiating interface with the receiving integrity identification being incomplete.

[0095] In this step, the abnormal receiving state refers to the state of the query initiating interface with problems in receiving the interface compatible data; the preset time threshold refers to the time critical value for judging whether the receiving feedback time is abnormal; and the abnormal query initiating interface refers to the query initiating interface in the abnormal receiving state.

[0096] In the embodiment of the present application, the preset time threshold is called to analyze the receiving state information of each query initiating interface; if the receiving feedback time of a certain query initiating interface exceeds the preset time threshold, it indicates that the data received by the interface is delayed, and it is determined that the interface is in the abnormal receiving state; if the receiving integrity identification of a certain query initiating interface is incomplete, it indicates that the data received by the interface is missing, and it is also determined that the interface is in the abnormal receiving state; all the query initiating interfaces in the abnormal receiving state are screened out to form the list of abnormal query initiating interfaces, and the abnormal interfaces needing to be processed are determined.

[0097] Step 404: the corresponding interface compatible data is sent to all abnormal query initiation interfaces to update the receiving state information of all abnormal query initiation interfaces until all abnormal query initiation interfaces are updated to normal query initiation interfaces.

[0098] In the embodiment of the present application, the compatible data corresponding to the abnormal query initiation interface is screened from the interface compatible data; the corresponding compatible data is re-sent to the abnormal query initiation interface through digital information transmission technology; after the sending is completed, the receiving feedback time and the receiving integrity identifier of the abnormal query initiation interface are re-recorded, and the receiving state information is updated; the updated receiving state information is verified, and if the receiving feedback time still exceeds the preset time threshold or the receiving integrity identifier is incomplete, the above re-sending and state updating operation is repeated until the receiving feedback time of all abnormal query initiation interfaces does not exceed the preset time threshold and the receiving integrity identifier is complete, that is, all abnormal query initiation interfaces are updated to normal query initiation interfaces.

[0099] Step 405: an interactive synchronization confirmation request is sent to all normal query initiation interfaces, and after receiving the interactive synchronization confirmation responses of all normal query initiation interfaces, the query intentions corresponding to each normal query initiation interface are extracted, and consistency check is performed on all query intentions, and the query intentions that pass the check are integrated into a unified query instruction.

[0100] In this step, the interactive synchronization confirmation response refers to the confirmation signal fed back by the normal query initiation interface after receiving the interactive synchronization confirmation request; the query intention refers to the user query purpose corresponding to the query initiation interface.

[0101] In the embodiment of the present application, an interactive synchronization confirmation request is sent to all normal query initiation interfaces, which is used to confirm whether each interface has accurately received and understood the semantic analysis result; after receiving the request, if each normal query initiation interface has accurately received and understood, it feeds back the interactive synchronization confirmation response that has been synchronized; after receiving the interactive synchronization confirmation responses of all normal query initiation interfaces, the query purpose of the user is extracted from the semantic analysis result corresponding to each normal query initiation interface to obtain the query intention corresponding to each normal query initiation interface; consistency check is performed on all query intentions to determine whether the query intentions of different interfaces are around the same core demand, and if they are consistent, it is determined that the check passes; all query intentions that pass the check are integrated to remove repeated contents to form a unified query instruction that can cover the query requirements of all interfaces.

[0102] The embodiment of the present application solves the problems of incompatible, abnormal reception and non-uniform query intention of multi-interface data transmission, ensures the reliable transmission and synchronization of the semantic analysis result among multiple interfaces, and provides a unified instruction basis for subsequent accurate data query.

[0103] The application provides one embodiment, step 105, determining the data source corresponding to each query field information in the semantic analysis result, specifically comprising the following steps: Step 501: constructing a field-data source correspondence table based on the correspondence between fields and data sources in historical query records.

[0104] In this step, the historical query record refers to a record containing query field and corresponding data source information stored in the past; the field-data source correspondence table refers to a table used to determine the correspondence between fields and data sources constructed based on historical query records; and the field label refers to a label used to identify different query fields in the field-data source correspondence table.

[0105] In the embodiment of the application, the stored historical query records are called, the corresponding data is sorted, classified by query field, and a dedicated field label is set for each query field; the field label, the corresponding query field name, and all data source information under the field are arranged in rows to form a field-data source correspondence table, so that the corresponding data source can be quickly located through the field label.

[0106] Step 502: extracting multiple query field information from the semantic analysis result, matching each query field information with the field label in the field-data source correspondence table, obtaining multiple candidate data sources corresponding to each query field information, and extracting at least one candidate data source from each query field information to form multiple candidate data source groups.

[0107] In this step, the candidate data source refers to a potential data source matched with the query field information; and the candidate data source group refers to a set formed by combining at least one candidate data source extracted from each query field information.

[0108] In the embodiment of the application, the semantic analysis result is subjected to field information extraction, and multiple query field information contained therein is identified; each query field information is compared with the field label in the field-data source correspondence table one by one, the field label matched with the query field information is found, and all data sources under the field label are obtained, which are multiple candidate data sources corresponding to each query field information; at least one candidate data source is selected from each query field information, and the selected candidate data sources are combined to form multiple candidate data source groups.

[0109] Step 503: calculating the overall field coverage of each candidate data source group relative to all query field information, and marking the candidate data source group with the largest overall field coverage as the target data source group; if there are multiple target data source groups, the target data source group with the least number of candidate data sources is retained.

[0110] In this step, the overall field coverage refers to the degree that the candidate data source set can cover all query field information; the target data source set refers to the candidate data source set with the maximum overall field coverage value and the least number of candidate data sources.

[0111] In the embodiment of the present application, for each candidate data source set, the number of query field information that can be covered by all candidate data sources in the set is counted; the overall field coverage is calculated as (the number of query field information covered by the candidate data source set / the total number of query field information) x 100%; the overall field coverage values of all candidate data source sets are compared, and the candidate data source set corresponding to the maximum overall field coverage value is preliminarily marked as the target data source set; if there are multiple candidate data source sets with the same maximum overall field coverage value, the number of candidate data sources in these candidate data source sets is counted, and the candidate data source set with the least number of candidate data sources is retained as the final target data source set.

[0112] Step 504: Marking the candidate data sources in the target data source set as the data sources corresponding to each query field information in the semantic analysis result.

[0113] In the embodiment of the present application, after determining the target data source set, all candidate data sources in the set are extracted; each candidate data source is associated with the corresponding query field information; through the association operation, the specific data source corresponding to each query field information is determined, and finally the candidate data sources in the target data source set are determined as the data sources corresponding to each query field information in the semantic analysis result.

[0114] The embodiment of the present application solves the problem of chaotic data source matching and incomplete coverage in multiple query fields, ensures that the selected data sources can accurately cover all query requirements, and provides efficient and adaptive data source support for subsequent unified query instruction data retrieval.

[0115] The present application provides a specific embodiment, step 106, according to the preset retrieval rule of each data source, data matching retrieval is performed on the unified query instruction, and an initial query result is obtained, so as to perform multi-interface cooperative natural language data query according to the initial query result, which specifically includes the following steps: Step 601: Extracting retrieval keywords from the unified query instruction, and converting the retrieval keywords into adaptive retrieval statements according to the preset retrieval rule of each data source.

[0116] In this step, the retrieval keywords refer to the core vocabulary extracted from the unified query instruction for data retrieval; the adaptive retrieval statement refers to the retrieval statement that can be recognized by the target data source after conversion according to the preset retrieval rule of each data source; the target data source refers to the data source that has been determined to correspond to each query field information.

[0117] In the embodiment of the present application, the unified query instruction is subjected to vocabulary splitting, and the vocabulary capable of reflecting the core requirement of the query is screened out, which is the retrieval keyword; the preset retrieval rule of each target data source is called; the retrieval keyword is converted into an adaptive retrieval statement conforming to the retrieval syntax of the data source according to the preset retrieval rule of the different target data source, so as to ensure that the adaptive retrieval statement can be recognized by the corresponding target data source.

[0118] Step 602: The adaptive retrieval statement is sent to the corresponding target data source, and the adaptive retrieval statement is matched with the data entries stored in the target data source, so as to obtain a plurality of matching data entries containing at least one retrieval keyword.

[0119] In this step, the data entry refers to a single data record stored in the target data source; and the matching data entry refers to a data entry in the target data source which is matched with the adaptive retrieval statement and contains at least one retrieval keyword.

[0120] In the embodiment of the present application, each adaptive retrieval statement is sent to the corresponding target data source according to the correspondence between the target data source and the adaptive retrieval statement; after the target data source receives the adaptive retrieval statement, the internal retrieval module is called to compare the content of the adaptive retrieval statement with all the data entries stored in the data source; and the data entry containing at least one retrieval keyword is screened out, which is the matching data entry.

[0121] Step 603: The relevance of the matching data entry to the unified query instruction is calculated, so as to arrange all the matching data entries according to the high-low order of the relevance, and obtain the arrangement result.

[0122] In this step, the relevance refers to the quantitative value of the close degree of association between the matching data entry and the unified query instruction; the high-low order refers to the arrangement order from large to small according to the relevance value; and the arrangement result refers to the result formed after all the matching data entries are arranged according to the high-low order of the relevance.

[0123] In the embodiment of the present application, for each matching data entry, the keyword contained therein is extracted and compared with the retrieval keyword of the unified query instruction, and the relevance is calculated as follows: relevance = (the number of coincident keywords of the matching data entry and the unified query instruction / the total number of retrieval keywords of the unified query instruction) x 100%; the relevance values of all the matching data entries are compared, and the matching data entries are sorted according to the high-low order from large to small according to the value, and the arrangement result is obtained after the sorting is completed, so as to ensure that the matching data entry with higher relevance is in a more forward position.

[0124] Step 604: The first preset number of matching data entries in the arrangement result are retained, and the initial query result is formed.

[0125] In this step, the front preset quantity refers to a preset quantity of matching data entries to be reserved; and the initial query result refers to a result formed after the front preset quantity of matching data entries in the arranged result is reserved.

[0126] In the embodiments of the present application, the front preset quantity is called according to the preset; the arranged result is viewed, and the front preset quantity of matching data entries is selected from the first matching data entry of the arranged result; the content of the selected matching data entries is integrated, and the repeated information is removed to ensure the uniqueness and conciseness of the information, and finally the initial query result is formed.

[0127] Step 605: According to the output format requirements of each query initiation interface, the initial query result is format-converted to obtain text format output data, voice format output data and touch interactive format output data.

[0128] In this step, the output format requirement refers to the specific requirement of each query initiation interface on the format of output data; the text format output data refers to data meeting the output format requirement of the text input interface; the voice format output data refers to data meeting the output format requirement of the voice input interface; and the touch interactive format output data refers to data meeting the output format requirement of the touch selection interface.

[0129] In the embodiments of the present application, the output format requirements of each query initiation interface are determined; and according to the output format requirements, the initial query result is format-converted, the initial query result is converted into text format output data in text format, into voice format output data in audio + text subtitle format, and into touch interactive format output data in clickable interactive card format, so as to ensure that the three types of converted data can be recognized and output by the corresponding query initiation interface.

[0130] Step 606: The text format output data, the voice format output data and the touch interactive format output data are respectively fed back to the corresponding query initiation interface to complete the multi-interface cooperative natural language data query.

[0131] In the embodiments of the present application, according to the correspondence between the query initiation interface and the output data, the text format output data is fed back to the text input interface, the voice format output data is fed back to the voice input interface, and the touch interactive format output data is fed back to the touch selection interface; after each query initiation interface receives the corresponding output data, the output data is presented to the user in the display or playing mode of the query initiation interface, and the user can obtain the initial query result through each interface, thereby completing the multi-interface cooperative natural language data query.

[0132] The embodiment of the application solves the problems of poor adaptability of multi-data source retrieval, low correlation of query results and incompatible multi-interface output formats, ensures the accuracy of data retrieval and the adaptability of multi-interface output, and improves the experience of users in obtaining query results through multi-interfaces.

[0133] Figure 3 A structural schematic diagram of one specific embodiment of a multi-interface cooperative natural language data query system provided by the embodiment of the application is shown in Figure 3 The system can include: The acquisition module 21 is configured to acquire text query data transmitted by a text input interface, voice query data transmitted by a voice input interface and touch instruction data transmitted by a touch selection interface. The optimization module 22 is configured to perform optimization processing on the voice query data and the touch instruction data respectively to obtain voice text data and touch text data, and to integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fusion text set. The analysis module 23 is configured to perform semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result. The sending module 24 is configured to send the semantic analysis result to each query initiation interface through a digital information transmission technology, and to process an abnormal query initiation interface in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have received an interaction synchronization confirmation request, and then generate a unified query instruction. The determination module 25 is configured to determine data sources corresponding to each query field information in the semantic analysis result. The retrieval module 26 is configured to perform data matching retrieval on the unified query instruction according to a preset retrieval rule of each data source to obtain an initial query result, and to perform multi-interface cooperative natural language data query according to the initial query result.

[0134] The multi-interface cooperative natural language data query system of the embodiment of the application is used to implement the multi-interface cooperative natural language data query method described above, and thus the specific embodiments of the multi-interface cooperative natural language data query system can be seen from the embodiment part of the multi-interface cooperative natural language data query method described above. The specific embodiments can refer to the descriptions of the respective embodiment parts, and will not be described here again.

[0135] The application further provides an electronic device including a memory configured to store a computer program and a processor configured to implement the steps of the multi-interface cooperative natural language data query method described above when executing the computer program.

[0136] The application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0137] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory, a random access memory, a mobile hard disk, a magnetic disk or an optical disk, and various media capable of storing a computer program.

[0138] The embodiments of the application further provide a computer program product, and the computer program product includes a computer program, and the computer program is executed by a processor to implement the steps in the method embodiments of the multi-interface cooperative natural language data query method.

[0139] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0140] The above describes in detail the method and system of the multi-interface cooperative natural language data query method provided by the application. The principles and implementation modes of the application are described by using specific examples. The above example description is only used to help understand the method and its core idea. It should be pointed out that for ordinary skilled person in the art, without departing from the principles of the application, the application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the application.

Claims

1. A method for multi-interface coordinated natural language data query, characterized in that, The method comprises the following steps: acquiring text query data transmitted by a text input interface, voice query data transmitted by a voice input interface, and touch instruction data transmitted by a touch selection interface; optimizing the voice query data and the touch instruction data respectively to obtain voice text data and touch text data, integrating the text query data, the voice text data, and the touch text data to obtain a multi-interface fusion text set; performing semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result; sending the semantic analysis result to each query initiation interface through digital information transmission technology, processing abnormal query initiation interfaces in each query initiation interface, and generating a unified query instruction after confirming that all query initiation interfaces are normal query initiation interfaces and have all received interaction synchronization confirmation requests; determining data sources corresponding to each query field information in the semantic analysis result; performing data matching retrieval on the unified query instruction according to a preset retrieval rule of each data source to obtain an initial query result, and performing multi-interface collaborative natural language data query according to the initial query result.

2. The method of claim 1, wherein, The method of optimizing the voice query data and the touch instruction data respectively to obtain voice text data and touch text data, integrating the text query data, the voice text data, and the touch text data to obtain a multi-interface fusion text set comprises the following steps: extracting acoustic features from the voice query data, combining historical voice samples, and constructing a voice matching relationship, wherein the acoustic features include frequency features, amplitude features, and time length features; dividing the voice query data into multiple text data segments based on the acoustic features and the voice matching relationship, performing semantic coherent splicing processing on pause interval parts in the text data segments, and performing voice correction processing on fuzzy pronunciation parts to obtain voice text data; extracting coordinate sequences from the touch instruction data, wherein the coordinate sequences include starting coordinate points, ending coordinate points, and intermediate coordinate points; determining function descriptions corresponding to each coordinate point in the coordinate sequences based on a preset function area division table, converting each function description into corresponding instruction text data segments, performing deduplication processing on repeated instruction data in the instruction text data segments, and filtering effective instruction data in conflicting instruction data in the instruction text data segments according to a preset priority rule to form touch text data; arranging and interface marking the text query data, the voice text data, and the touch text data according to the order of a first timestamp of the text query data, a second timestamp of the voice text data, and a third timestamp of the touch text data to obtain a multi-interface fusion text set.

3. The method of claim 1, wherein, The method of performing semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result comprises the following steps: extracting multiple standard semantic items, multiple semantic relationship labels, and multiple field classification labels from the preset natural language semantic analysis annotation data set; According to the semantic pause mark of the natural language, the multi-interface fusion text set is divided into a plurality of semantic segments, and a matching degree of each semantic segment with each standard semantic item is calculated; The standard semantic item corresponding to the maximum matching degree in each semantic segment is taken as a target semantic item of the corresponding semantic segment, and the target semantic item corresponding to each semantic segment, a semantic relationship label, and a domain classification label are associated to obtain a plurality of labeled semantic segments; According to the original data sequence of the multi-interface fusion text set, all the labeled semantic segments are arranged to obtain a semantic segment set; According to the semantic relationship labels of adjacent pairs of labeled semantic segments in the semantic segment set, an association logic of each labeled semantic segment in the semantic segment set is constructed; The target semantic item, the semantic relationship label, the domain classification label, and the association logic are integrated to form a semantic analysis result.

4. The method of claim 3, wherein, According to the semantic relationship labels of adjacent pairs of labeled semantic segments in the semantic segment set, an association logic of each labeled semantic segment in the semantic segment set is constructed, including: The semantic relationship labels of all adjacent pairs of labeled semantic segments in the semantic segment set are parsed to obtain an association type of each adjacent pair of labeled semantic segments, and the association type includes a cause-and-effect relationship, a parallel relationship, and a progressive relationship; The matching degrees of each association type and a preset relationship type rule are calculated, and the association type corresponding to the adjacent pair of labeled semantic segments with a matching degree exceeding a preset matching threshold is added with an association type identifier to form a segment relationship chain; Based on a preset association type hierarchy rule, the association types in the segment relationship chain are hierarchically divided to obtain association types of different priorities; The association types of different priorities are compared with a preset priority to obtain a priority comparison result, and each adjacent pair of labeled semantic segments in the segment relationship chain is rearranged according to the priority comparison result to obtain an adjusted segment relationship chain; According to the adjusted segment relationship chain, the association logic of the semantic segment set is constructed.

5. The method of claim 1, wherein, Through digital information transmission technology, the semantic analysis result is sent to each query initiation interface, and abnormal query initiation interfaces in each query initiation interface are processed until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have received an interaction synchronization confirmation request, and a unified query instruction is generated, including: Based on the transmission protocol of each query initiation interface, the semantic analysis result is data-converted to obtain interface-compatible data, and the interface-compatible data includes first compatible data corresponding to a text input interface, second compatible data corresponding to a voice input interface, and third compatible data corresponding to a touch selection interface; Through digital information transmission technology, the interface-compatible data is sent to the corresponding query initiation interface, and the receiving state information of each query initiation interface is recorded, including the receiving feedback time and the receiving integrity identifier; According to the receiving state information, an abnormal query initiation interface is identified as an abnormal receiving state, the abnormal query initiation interface includes a query initiation interface whose receiving feedback time exceeds a preset time threshold or a query initiation interface whose integrity identifier is incomplete; Corresponding interface compatible data is sent to all abnormal query initiation interfaces to update the receiving state information of all abnormal query initiation interfaces until all abnormal query initiation interfaces are updated to normal query initiation interfaces; An interaction synchronization confirmation request is sent to all normal query initiation interfaces, and after receiving the interaction synchronization confirmation responses of all normal query initiation interfaces, the query intent corresponding to each normal query initiation interface is extracted, and consistency verification is performed on all query intents, and the query intents that pass the verification are integrated into a unified query instruction.

6. The method of claim 1, wherein, Determine the data source corresponding to each query domain information in the semantic analysis result, including: Based on the correspondence between the fields and data sources in the historical query records, a field data source correspondence table is constructed; From the semantic analysis result, a plurality of query domain information is extracted, each query domain information is matched with the field label in the field data source correspondence table, a plurality of candidate data sources corresponding to each query domain information are obtained, at least one candidate data source is extracted from each query domain information, to form a plurality of candidate data source groups; Calculate the overall field coverage of each candidate data source group with respect to all query domain information, and mark the candidate data source group corresponding to the largest overall field coverage as the target data source group, if there are multiple target data source groups, keep the target data source group with the least number of candidate data sources; Mark the candidate data sources in the target data source group as the data sources corresponding to each query domain information in the semantic analysis result.

7. The method of claim 1, wherein, According to the preset retrieval rule of each data source, data matching retrieval is performed on the unified query instruction to obtain an initial query result, so as to perform multi-interface cooperative natural language data query according to the initial query result, including: Extract the retrieval keywords from the unified query instruction, and convert the retrieval keywords into adaptive retrieval sentences according to the preset retrieval rule of each data source; Send the adaptive retrieval sentence to the corresponding target data source, and match the adaptive retrieval sentence with the data entries stored in the target data source to obtain a plurality of matching data entries containing at least one retrieval keyword; Calculate the relevance of the matching data entries and the unified query instruction, and arrange all matching data entries according to the order of the relevance to obtain an arrangement result; Keep the first preset number of matching data entries in the arrangement result to form an initial query result; According to the output format requirement of each query initiation interface, the initial query result is format-converted to obtain text format output data, voice format output data and touch interaction format output data; The text format output data, the voice format output data and the touch interaction format output data are respectively fed back to corresponding query initiation interfaces to complete the multi-interface cooperative natural language data query.

8. A multi-interface coordinated natural language data query system, characterized in that, The method comprises the steps of: An acquisition module is configured to acquire text query data transmitted by a text input interface, voice query data transmitted by a voice input interface and touch instruction data transmitted by a touch selection interface; An optimization module is configured to perform optimization processing on the voice query data and the touch instruction data respectively to obtain voice text data and touch text data, and to integrate the text query data, the voice text data and the touch text data to obtain a multi-interface fusion text set; An analysis module is configured to perform semantic analysis on the multi-interface fusion text set based on a preset natural language semantic analysis annotation data set to obtain a semantic analysis result; A sending module is configured to send the semantic analysis result to each query initiation interface through a digital information transmission technology, and to process an abnormal query initiation interface in each query initiation interface until it is confirmed that all query initiation interfaces are normal query initiation interfaces and have all received interaction synchronization confirmation requests, and then to generate a unified query instruction; A determination module is configured to determine data sources corresponding to each query field information in the semantic analysis result; A retrieval module is configured to perform data matching retrieval on the unified query instruction according to a preset retrieval rule of each data source to obtain an initial query result, and to perform multi-interface cooperative natural language data query according to the initial query result.

9. An electronic device, comprising: The method comprises the steps of: A memory is configured to store a computer program; A processor is configured to implement the steps of the multi-interface cooperative natural language data query method according to any one of claims 1 to 7 when the computer program is executed.

10. A computer-readable storage medium, characterized in that, The computer program stored in the computer readable storage medium can implement the multi-interface cooperative natural language data query method according to any one of claims 1 to 7 when executed by the processor.

Images