Method for controlling large-screen data index change display through voice interaction
By performing domain semantic enhancement parsing and scenario adaptation on voice commands, the problem of inaccurate data operation on large screens was solved, and accurate control and display of large screen data indicators were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AIPARK TECHNOLOGY CO LTD
- Filing Date
- 2025-12-18
- Publication Date
- 2026-05-01
AI Technical Summary
In existing technologies, voice commands cannot be accurately parsed in large-screen environments, leading to inaccurate data operations and affecting the accuracy and real-time performance of data presentation.
By performing domain semantic enhancement parsing on the original voice commands, combining the current application scenario and user identity information for permission verification and scenario adaptation, adapted operation commands are generated, and linkage control and visualization rendering are executed, outputting multimodal feedback signals.
It improves the accuracy of voice command parsing, enables accurate control and display of large-screen data indicators, and ensures the accuracy and real-time performance of data operations.
Smart Images
Figure CN121963719A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and specifically to a method for controlling the display of changes in data indicators on a large screen through voice interaction. Background Technology
[0002] With the widespread application of large screens in command centers, exhibition halls, and enterprise business analysis, voice interaction is gradually becoming an important means to replace traditional mouse and touch control. However, large screen environments are often characterized by complex background noise, significant spatial reverberation, and diverse industry terminology, making it easy for voice content to become ambiguous during recognition, and difficult to accurately extract user intent, target data indicators, and operation types. Due to inaccurate voice parsing, large screen systems are prone to deviations when performing data operations such as filtering, drill-down, comparison, or visualization adjustments, resulting in inconsistencies between the displayed content and user needs, affecting the accuracy and real-time performance of data presentation. Summary of the Invention
[0003] This application provides a method for controlling the display of large-screen data indicators through voice interaction, which is used to address the technical problem of inaccurate large-screen data operation caused by the inability to accurately interpret voice commands in existing technologies.
[0004] In view of the above problems, this application provides a method for controlling the display of large-screen data indicators through voice interaction, the method comprising:
[0005] The original voice command is recognized and parsed to obtain the parsing result, which includes the user intent, target data indicators, and target operation type determined through domain semantic enhancement. Based on the current application scenario and user identity information, the parsing result is subjected to permission verification and scenario adaptation to generate an adapted operation command. The adapted operation command is executed to perform linkage control and visualization rendering on the target data indicators and update the large screen display status. Based on the updated large screen display status, a multimodal feedback signal is generated and output.
[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0007] This application recognizes and analyzes the original voice command to obtain an analysis result, which includes the user intention, target data metrics, and target operation type determined through domain semantic enhancement; based on the current application scenario and user identity information, it performs permission verification and scenario adaptation on the analysis result to generate an adapted operation command; executes the adapted operation command to perform linkage control and visual rendering on the target data metrics, and updates the large screen display status; based on the updated large screen display status, generates and outputs a multi-modal feedback signal. The present invention solves the technical problem that the inaccurate operation of large screen data caused by the inability to accurately parse voice commands in the prior art. By performing domain semantic enhancement analysis on voice commands and combining scenario and user identity for adaptation and execution, it achieves the technical effect of improving the accuracy of command parsing and realizing the accurate control of large screen data metrics. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0009] Figure 1 Schematic flowchart of a method for controlling the change display of large screen data metrics through voice interaction provided by an embodiment of this application;
[0010] Figure 2 Schematic flowchart of the process for obtaining the analysis result in a method for controlling the change display of large screen data metrics through voice interaction provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0011] This application provides a method for controlling the change display of large screen data metrics through voice interaction. Aiming at solving the technical problem that the inaccurate operation of large screen data caused by the inability to accurately parse voice commands in the prior art, by performing domain semantic enhancement analysis on voice commands and combining scenario and user identity for adaptation and execution, it achieves the technical effect of improving the accuracy of command parsing and realizing the accurate control of large screen data metrics.
[0012] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by this application.
[0013] It should be noted that any variation of the terms "comprising" and "having" is intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.
[0014] Examples, such as Figure 1 As shown, this application provides a method for controlling the display of data indicators on a large screen via voice interaction, the method comprising:
[0015] Step S100: Recognize and parse the original voice command to obtain the parsing result, which includes the user intent, target data indicators and target operation type determined through domain semantic enhancement.
[0016] In this embodiment, when recognizing and parsing the original voice command, the corresponding industry-specific dictionary is first loaded according to the current application scenario. Then, the automatic speech recognition model uses this industry-specific dictionary to perform acoustic decoding and text conversion on the original voice command, obtaining the recognized text. Afterwards, based on the contextual semantic cache, the semantic understanding model performs semantic understanding on the recognized text and fills the intent slots according to a preset semantic structure. The intent slots include indicator slots, operation slots, and parameter slots. The indicator slots are used to determine the target data indicators involved in the voice command. Target data indicators refer to business data fields that can be displayed, filtered, or analyzed in the large-screen system, such as sales volume, inventory level, customer traffic, and equipment operating efficiency. Through the above structured parsing, a final parsing result containing the user intent, target data indicators, and target operation type is generated.
[0017] Furthermore, such as Figure 2 As shown, the method provided in the application embodiment, in addition to recognizing and parsing the original voice command to obtain the parsing result, further includes:
[0018] Load the corresponding industry domain dictionary according to the current application scenario; use the industry domain dictionary to perform speech recognition on the original voice command through an automatic speech recognition model to obtain the recognized text; perform semantic understanding on the recognized text through a semantic understanding model based on the context semantic cache, and fill the predefined intent slots to generate the parsing result, wherein the intent slots include at least indicator slots, operation slots and parameter slots.
[0019] In this embodiment, the corresponding industry-specific dictionary is first loaded based on the current application scenario. Specifically, when a user enters a scenario, the scenario identifier is read, such as a showroom explanation, emergency command, or equipment monitoring scenario. Based on this scenario, relevant professional terms are selected from a pre-set industry corpus resource, and these selected terms are combined to form the industry-specific dictionary. For example, in the retail industry scenario, the industry-specific dictionary includes terms such as sales volume, average order value, and year-on-year comparison.
[0020] Subsequently, the original voice commands are processed by an automatic speech recognition model using an industry-specific dictionary. This process begins by acquiring the user's voice signal and extracting acoustic features, which are then input into the automatic speech recognition model. This model, trained on a large amount of voice data, uses CTC decoding to progressively align the acoustic features into a text sequence. During decoding, an industry-specific dictionary is introduced as a linguistic constraint, causing the automatic speech recognition model to prioritize industry-specific terms among multiple candidate words. For example, if a user says "show sales figures for the first half of 2024," the generated recognition text will prioritize outputting sales figures. Through this process, speech recognition is completed, and the recognized text is obtained.
[0021] Next, based on the contextual semantic cache, the identified text is semantically understood using a semantic understanding model, and predefined intent slots are filled. In this process, the identified text and the contextual semantic cache are input together into the semantic understanding model trained on the task corpus, and the Transformer-CRF sequence labeling method is used to complete intent discrimination and slot filling. The contextual semantic cache refers to semantic information already confirmed in the previous round or the current session. For example, when the identified text is a comparison of sales in the first half of 2024 and the first half of 2023, the semantic understanding model will fill the indicator slot with sales, the operation slot with the comparison, and the parameter slots with the first half of 2024 and the first half of 2023. After this step, a parsing result containing user intent, target data indicators, and target operation types is obtained.
[0022] Furthermore, the method provided in the application embodiments also includes:
[0023] The target operation type is any one or a combination of data filtering, data drill-down, multi-indicator comparison, indicator linkage, and visualization form adjustment.
[0024] In this embodiment, the target operation type is a voice-triggered operation method in a large-screen data analysis scenario, including data filtering, data drill-down, multi-indicator comparison, indicator linkage, and visualization adjustment. Data filtering is used to extract sub-data that meet certain conditions from a complete dataset, such as filtering sales revenue for the first half of 2024; data drill-down is used to view more granular data layer by layer along a dimension hierarchy, such as drilling down to the city dimension; multi-indicator comparison is used to compare the trends or values of multiple indicators in the same view, such as comparing sales volume and inventory levels; indicator linkage is used to automatically retrieve other related indicators when operating on a certain indicator, such as automatically linking gross profit margin when displaying sales volume; visualization adjustment is used to change the presentation of charts, such as switching to a line chart.
[0025] Step S200: Based on the current application scenario and user identity information, perform permission verification and scenario adaptation on the parsing result, and generate adapted operation instructions.
[0026] Furthermore, in the method provided in the application embodiments, based on the current application scenario and user identity information, the method further includes performing permission verification and scenario adaptation on the parsing result, and includes:
[0027] Obtain the corresponding set of operation permissions based on the user identity information; determine whether the target operation type in the parsing result belongs to the set of operation permissions; if so, adjust the parsing priority or recommend related indicators for the target data indicators in the parsing result according to the scenario rule library bound to the current application scenario, and generate the adapted operation instructions.
[0028] In this embodiment of the application, when performing permission verification and scenario adaptation on the parsing results based on the current application scenario and user identity information, the corresponding set of operation permissions is first obtained according to the user identity information. The role mapping method is used to read the executable operation entries from the preset permission configuration according to the identity identifier, load them in a fixed order and remove duplicates, forming a set of operation permissions with the target operation type as the key.
[0029] Next, it is determined whether the target operation type in the parsing result belongs to the operation permission set. That is, the target operation type is compared with the items in the operation permission set one by one to generate a judgment flag for permission or prohibition, and obtain the permission verification result used for scene adaptation branch selection.
[0030] If deemed permissible, the parsing priority of the target data indicators in the parsing results is adjusted or associated indicators are recommended based on the scenario rule base bound to the current application scenario. The scenario rule base is loaded by the scenario identifier using the rule matching method, and the priority rearrangement subprocess is called to perform a single rearrangement of the parsing order of the target data indicators, or the association recommendation subprocess is called to output a set of associated indicators according to the preset indicator association relationship, thus obtaining an intermediate configuration containing the parsing priority adjustment result or the associated indicator recommendation result.
[0031] Finally, based on the intermediate configuration, the adapted operation instructions are generated. The instruction assembly method is used, with the permission verification result as a constraint and the parsing priority adjustment result or the related indicator recommendation result as the content. The execution parameters of the target operation type and the target data indicator are written in the order of the predefined fields, and the adapted operation instructions are output.
[0032] Step S300: Execute the adapted operation instructions to perform linkage control and visualization rendering of the target data indicators and update the display status of the large screen.
[0033] Furthermore, the method provided in the application embodiments, which involves linkage control and visualization rendering of the target data indicators, further includes:
[0034] Based on the target data indicators in the adapted operation instructions, the preset data linkage rules are queried to obtain the associated data indicators; based on the target data indicators and the associated data indicators, a combined operation instruction is generated; the combined operation instruction is sent to the visualization engine to drive the large screen to perform data query and graphics rendering, and obtain the updated large screen display status.
[0035] In this embodiment, when executing the adapted operation instruction to perform linked control and visualization rendering of the target data indicator, the target data indicator identifier in the adapted operation instruction is first read. Then, this identifier is used as a search condition, and each item is compared against predefined data linkage rules. Rule entries that have business or logical connections with the target data indicator are selected based on the matching conditions. After matching is completed, indicator items related to the target data indicator are extracted from the selected rule entries, thereby obtaining the associated data indicator.
[0036] Next, based on the target data metric and related data metrics, a template-filling method is used to generate combined operation instructions. In this process, the target data metric is the primary operation object, and related data metrics are secondary operation objects. Data query parameters, operation type parameters, and visualization parameters are sequentially filled in according to the predefined instruction structure, and the parameters are completed by combining the time range and conditional constraints from the adapted operation instructions. After filling, a consistency check is performed on the generated instruction content, forming a combined operation instruction that can simultaneously drive the processing of multiple metrics.
[0037] Finally, the combined operation instructions are sent to the visualization engine. The visualization engine, based on the data query parameters included in the combined operation instructions, executes a data query from the data source to obtain the data content corresponding to the target data indicator and related data indicators. After obtaining the data, the visualization engine then performs graphic rendering according to the visualization parameters recorded in the combined operation instructions, including steps such as chart type selection, coordinate system generation, data element drawing, and key data annotation. The rendered graphic results are then updated to the large screen display area, thus forming an updated large screen display state encompassing the target data indicator and related data indicators.
[0038] Step S400: Based on the updated large screen display status, generate and output multimodal feedback signals.
[0039] Furthermore, the method provided in the application embodiments, which generates and outputs a multimodal feedback signal based on the updated large-screen display state, further includes:
[0040] Generate a voice broadcast text describing the updated large screen display status and output it through speech synthesis; simultaneously generate a visual feedback interface in a preset area of the large screen, the visual feedback interface including operation confirmation information and key data annotations in the updated large screen display status; the voice broadcast and the visual feedback interface together constitute the multimodal feedback signal.
[0041] In this embodiment, based on the updated large-screen display status, a voice broadcast text is first generated. During this process, information is extracted from the updated large-screen display status, identifying the target data indicators, related data indicators, and key data annotations involved, and forming descriptive statements according to preset text generation rules. For example, when the growth rate data for a certain month belongs to the key data annotation content, the generated voice broadcast text will include a description of the growth rate for that month. The generated voice broadcast text is then processed through a speech synthesis step, converting the text content into an outputtable speech signal to obtain a speech output result for auditory presentation.
[0042] While generating the voice-over text, a visual feedback interface is also generated in a preset area on the large screen. During execution, the interface layout is determined based on the updated large screen display status, placing the completion notification in the designated area and visually presenting key data annotations, including numerical highlighting, color coding, or trend symbol annotations, to ensure clear visual recognition of the key data. The generated visual feedback interface is displayed as a layer overlaid on the current large screen content to provide clear visual prompts.
[0043] After both the voice output and the visual feedback interface are generated, they are processed for synchronous output to ensure that the auditory and visual feedback are consistent in time. For example, when the voice broadcast points to a key data point, the visual feedback interface simultaneously highlights that data point. After synchronization, the voice output and the visual feedback interface together constitute a multimodal feedback signal, realizing the synchronous presentation of the updated large-screen display status, thus obtaining the multimodal feedback signal.
[0044] Furthermore, the method provided in the application embodiments also includes:
[0045] Determine whether the adapted operation instruction is a preset high-risk type operation; if it is a preset high-risk type operation, generate a risk confirmation prompt and request user confirmation before execution; in response to the user's confirmation of execution, create an operation snapshot corresponding to the current large screen display state; execute the adapted operation instruction; if a preset abnormal condition is triggered during execution, roll back the large screen display state to the state before execution based on the operation snapshot.
[0046] In this embodiment of the application, after receiving the adaptation operation instruction, the adaptation operation instruction is first judged according to the preset high-risk type operation list to identify whether it involves high-impact actions such as deleting data, overwriting configuration, batch changing or affecting key views, and a high-risk judgment result is obtained.
[0047] If the operation is determined to be a pre-defined high-risk type, a risk confirmation prompt will be generated before execution and the user will be asked to confirm. The risk confirmation prompt is used to clarify the scope and consequences of the instruction to ensure that the operation is known and controllable.
[0048] After receiving the user's confirmation of execution, an operation snapshot corresponding to the current large screen display state is created. The operation snapshot is used to record elements such as interface layout, displayed target data indicators, rendering content of related data indicators, and key data annotations.
[0049] Then, the adaptation operation instructions are executed to complete the queries, processing, and rendering related to the target data indicators according to the established process, and update the display status of the large screen.
[0050] If a preset abnormal condition is triggered during execution, such as an error returned by the data source or an abnormal rendering process, the large screen display state will be rolled back to the state before execution based on the aforementioned operation snapshot, thereby ensuring that the large screen display can still be maintained under high-risk operation or abnormal scenario.
[0051] Furthermore, the method provided in the application embodiments also includes:
[0052] The system records interaction log data for voice interaction, which includes the original voice commands, the parsing results, the adapted operation commands, and user feedback information. Based on the interaction log data, the automatic speech recognition model and the semantic understanding model are iteratively optimized.
[0053] In this embodiment of the application, interaction log data is recorded during the voice interaction process. The original voice command, parsing result, adapted operation command and user feedback information are written in chronological order to ensure that one interaction generates one complete log. The log items include voice audio reference, recognized text snapshot, intent and slot structure, target data indicators and target operation type, execution result mark and user confirmation or correction content.
[0054] Next, the interaction log data is organized to establish a one-to-one correspondence between the original voice commands and parsing results, the adapted operation commands, and user feedback information. The log samples are divided into training sets and validation sets, and data samples for model training are output, which include the correct recognized text, the correct user intent, the correct indicator slots, operation slots, and parameter slot labels.
[0055] The automatic speech recognition model is optimized based on the processed data samples. The original speech commands are aligned with the corresponding correctly recognized text for training. Industry terms and high-frequency business words are highlighted and labeled. Output errors of difficult-to-distinguish near-homophones and domain words are corrected. Language constraints and word list weights are updated, and the updated automatic speech recognition model is output to improve the stability and accuracy of subsequent text recognition.
[0056] The semantic understanding model is optimized based on the same batch of data samples. The parsing structure is trained under supervision using the correct user intent and slot labels. Intent classification bias and slot filling defects are corrected, the binding relationship between target data indicators and target operation types is strengthened, and the updated semantic understanding model is output.
[0057] Through the above process, the automatic speech recognition model and semantic understanding model are iteratively optimized.
[0058] In summary, the embodiments of this application have at least the following technical effects:
[0059] This application recognizes and analyzes the original voice command to obtain an analysis result, which includes the user intention, target data index, and target operation type determined through domain semantic enhancement; based on the current application scenario and user identity information, it performs permission verification and scenario adaptation on the analysis result to generate an adapted operation command; executes the adapted operation command to perform linkage control and visual rendering on the target data index, and updates the large-screen display status; based on the updated large-screen display status, generates and outputs a multi-modal feedback signal. The present invention solves the technical problem that the large-screen data operation is inaccurate due to the inability to accurately parse voice commands in the prior art. By performing domain semantic enhancement parsing on voice commands and combining scenario and user identity for adaptation and execution, it achieves the technical effect of improving the accuracy of command parsing and realizing accurate control of large-screen data indicators.
[0060] It should be noted that the above sequence of embodiments of this application is only for description and does not represent the superiority or inferiority of the embodiments. And the above describes specific embodiments of this specification. The processes depicted in the drawings do not necessarily require the specific order and continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be beneficial.
[0061] The above are only the preferred embodiments of the present invention and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above with the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to the above-disclosed technical content to be equivalent embodiments within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.
Claims
1. A method for controlling the display of data indicators on a large screen via voice interaction, characterized in that, The method includes: The original voice command is recognized and parsed to obtain the parsing result, which includes the user intent, target data indicators and target operation type determined by domain semantic enhancement; Based on the current application scenario and user identity information, the parsing results are subjected to permission verification and scenario adaptation, and adapted operation instructions are generated. Execute the adapted operation instructions to perform linkage control and visualization rendering of the target data indicators and update the large screen display status. Based on the updated large screen display status, generate and output multimodal feedback signals.
2. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 1, characterized in that, The method for recognizing and parsing the original voice commands to obtain the parsing results includes: Load the corresponding industry-specific dictionary based on the current application scenario; The original voice command is recognized by an automatic speech recognition model using the industry-specific dictionary to obtain the recognized text. Based on the context semantic cache, the recognized text is semantically understood through a semantic understanding model, and predefined intent slots are filled to generate the parsing result. The intent slots include at least indicator slots, operation slots, and parameter slots.
3. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 1, characterized in that, The target operation type is any one or a combination of data filtering, data drill-down, multi-indicator comparison, indicator linkage, and visualization form adjustment.
4. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 1, characterized in that, Based on the current application scenario and user identity information, the parsing results are subjected to permission verification and scenario adaptation. The methods include: Obtain the corresponding set of operation permissions based on the user's identity information; Determine whether the target operation type in the parsing result belongs to the operation permission set; If so, then based on the scenario rule library bound to the current application scenario, the parsing priority of the target data indicators in the parsing result is adjusted or associated indicators are recommended, and the adapted operation instructions are generated.
5. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 4, characterized in that, The method for performing linked control and visualization rendering of the target data indicators includes: Based on the target data indicators in the adapted operation instructions, query the preset data linkage rules to obtain the associated data indicators; Based on the target data indicator and the associated data indicator, generate a combined operation instruction; The combined operation instructions are sent to the visualization engine to drive the large screen to perform data query and graphics rendering, and obtain the updated large screen display status.
6. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 1, characterized in that, Based on the updated large-screen display status, a multimodal feedback signal is generated and output, including the following methods: Generate a voice broadcast text describing the updated large screen display status and perform speech synthesis output; A visual feedback interface is generated synchronously in a preset area of the large screen. The visual feedback interface includes operation confirmation information and key data annotations in the updated large screen display status. The voice broadcast and the visual feedback interface together constitute the multimodal feedback signal.
7. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 1, characterized in that, The method also includes: Determine whether the adapted operation instruction is a preset high-risk type operation; If the operation is a preset high-risk type, a risk confirmation prompt will be generated and the user will be asked to confirm before execution; In response to the user's confirmed command, create an operation snapshot corresponding to the current large screen display state; Execute the adapted operation instructions; If a preset abnormal condition is triggered during execution, the large screen display state will be rolled back to the state before execution based on the operation snapshot.
8. The method for controlling the display of large-screen data indicators via voice interaction as described in claim 1, characterized in that, The method also includes: The interaction log data records the voice interaction, and the interaction log data includes the original voice command, the parsing result, the adapted operation command and user feedback information; Based on the interaction log data, the automatic speech recognition model and semantic understanding model are iteratively optimized.